Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

From: Justin Pryzby <pryzby(at)telsasoft(dot)com>
To: Fabien COELHO <coelho(at)cri(dot)ensmp(dot)fr>
Cc: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, David Steele <david(at)pgmasters(dot)net>, pgsql-hackers(at)postgresql(dot)org, "Bossart, Nathan" <bossartn(at)amazon(dot)com>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Subject: Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)
Date: 2020-05-03 02:42:15
Message-ID: 20200503024215.GJ28974@telsasoft.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sun, Apr 12, 2020 at 01:53:40PM +0200, Fabien COELHO wrote:
> About v15, seen as one patch.

Thanks for looking.

> - I'm wondering whether could pg_stat_file call pg_ls_dir_files without
> too much effort? ISTM that the output structure nearly the same. I do
> not like much having one function specialized for files and one for
> directories.

I refactored but not like that. As I mentioned in the commit message, I don't
see a good way to make a function operate on a file when the function's primary
data structure is a DIR*. Do you ? I don't think it should call stat() and
then conditionally branch off to pg_stat_file().

There are two functions because they wrap two separate syscalls, which see as
good, transparent goal. If we want a function that does what "ls -al" does,
that would also be a good example to follow, except that we already didn't
follow it.

/bin/ls first stat()s the path, and then either outputs its metadata (if it's a
file or if -d was specified) or lists a dir. It's essentially a wrapper around
*two* system calls (stat and readdir/getdents).

Maybe we could invent a new pg_ls() which does that, and then refactor existing
code. Or, maybe it would be a SQL function which calls stat() and then
conditionally calls pg_ls_dir if isdir=True (or type='d'). That would be easy
if we merge the commit which outputs all stat fields.

I'm still hoping for confirmation from a committer if this approach is worth
pursuing:

https://www.postgresql.org/message-id/20200310183037.GA29065%40telsasoft.com
https://www.postgresql.org/message-id/20200313131232.GO29065%40telsasoft.com
|Rather than making "recurse into tmpdir" the end goal:
|
| - add a function to show metadata of an arbitrary dir;
| - add isdir arguments to pg_ls_* functions (including pg_ls_tmpdir but not
| pg_ls_dir).
| - maybe add pg_ls_dir_recurse, which satisfies the original need;
| - retire pg_ls_dir (does this work with tuplestore?)
| - profit

--
Justin

Attachment Content-Type Size
v16-0001-Document-historic-behavior-of-links-to-directori.patch text/x-diff 1.1 KB
v16-0002-pg_stat_file-and-pg_ls_dir_-to-use-lstat.patch text/x-diff 2.2 KB
v16-0003-Add-tests-on-pg_ls_dir-before-changing-it.patch text/x-diff 2.0 KB
v16-0004-Add-pg_ls_dir_metadata-to-list-a-dir-with-file-m.patch text/x-diff 19.1 KB
v16-0005-pg_ls_tmpdir-to-show-directories-and-isdir-argum.patch text/x-diff 7.4 KB
v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch text/x-diff 9.4 KB
v16-0007-Add-pg_ls_dir_recurse-to-show-dir-recursively.patch text/x-diff 5.8 KB
v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch text/x-diff 3.1 KB
v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch text/x-diff 18.7 KB
v16-0010-pg_ls_-to-show-file-type-and-show-special-files.patch text/x-diff 17.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2020-05-03 03:09:13 Re: full-text search with GiST indexes seems to be getting slower since 9.1
Previous Message Andres Freund 2020-05-03 02:36:43 Re: design for parallel backup