Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

From: Justin Pryzby <pryzby(at)telsasoft(dot)com>
To: Fabien COELHO <coelho(at)cri(dot)ensmp(dot)fr>
Cc: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, David Steele <david(at)pgmasters(dot)net>, pgsql-hackers(at)postgresql(dot)org, "Bossart, Nathan" <bossartn(at)amazon(dot)com>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Subject: Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)
Date: 2020-03-31 20:08:12
Message-ID: 20200331200812.GI14618@telsasoft.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Mar 17, 2020 at 02:04:01PM -0500, Justin Pryzby wrote:
> > The example in the documentation could be better indented. Also, ISTM that
> > there are two implicit laterals (format & pg_ls_dir_recurse) that I would
> > make explicit. I'd use the pcs alias explicitely. I'd use meaningful aliases
> > (eg ts instead of b, …).
>
> > On reflection, I think that a boolean "isdir" column is a bad idea because
> > it is not extensible. I'd propose to switch to the standard "ls" approach of
> > providing the type as one character: '-' for regular, 'd' for directory, 'l'
> > for link, 's' for socket, 'c' for character special…
>
> I think that's outside the scope of the patch, since I'd want to change
> pg_stat_file; that's where I borrowed "isdir" from, for consistency.
>
> Note that both LS_DIR_HISTORIC and LS_DIR_MODERN include LS_DIR_SKIP_SPECIAL,
> so only pg_ls_dir itself show specials, so they way to do it would be to 1)
> change pg_stat_file to expose the file's "type", 2) use pg_ls_dir() AS a,
> lateral pg_stat_file(a) AS b, 3) then consider also changing LS_DIR_MODERN and
> all the existing pg_ls_*.

The patch intends to fix the issue of "failing to show failed filesets"
(because dirs are skipped) while also generalizing existing functions (to show
directories and "isdir" column) and providing some more flexible ones (to list
file and metadata of a dir, which is currently possible [only] for "special"
directories, or by recursively calling pg_stat_file).

I'm still of the opinion that supporting arbitrary file types is out of scope,
but I changed the "isdir" to show "type". I'm only supporting '[-dl]'. I
don't want to have to check #ifdef S_ISDOOR or whatever other vendors have. I
insist that it is a separate patch, since it depends on everything else, and I
have no feedback from anybody else as to whether any of that is desired.

template1=# SELECT * FROM pg_ls_waldir();
name | size | access | modification | change | creation | type
--------------------------+----------+------------------------+------------------------+------------------------+----------+------
barr | 0 | 2020-03-31 14:43:11-05 | 2020-03-31 14:43:11-05 | 2020-03-31 14:43:11-05 | | ?
baz | 4096 | 2020-03-31 14:39:18-05 | 2020-03-31 14:39:18-05 | 2020-03-31 14:39:18-05 | | d
foo | 0 | 2020-03-31 14:39:37-05 | 2020-03-31 14:39:37-05 | 2020-03-31 14:39:37-05 | | -
archive_status | 4096 | 2020-03-31 14:38:20-05 | 2020-03-31 14:38:18-05 | 2020-03-31 14:38:18-05 | | d
000000010000000000000001 | 16777216 | 2020-03-31 14:42:53-05 | 2020-03-31 14:43:08-05 | 2020-03-31 14:43:08-05 | | -
bar | 3 | 2020-03-31 14:39:16-05 | 2020-03-31 14:39:01-05 | 2020-03-31 14:39:01-05 | | l

Attachment Content-Type Size
v15-0001-Document-historic-behavior-of-links-to-directori.patch text/x-diff 1.1 KB
v15-0002-pg_stat_file-and-pg_ls_dir_-to-use-lstat.patch text/x-diff 2.2 KB
v15-0003-Add-tests-on-pg_ls_dir-before-changing-it.patch text/x-diff 2.0 KB
v15-0004-Add-pg_ls_dir_metadata-to-list-a-dir-with-file-m.patch text/x-diff 19.4 KB
v15-0005-pg_ls_tmpdir-to-show-directories-and-isdir-argum.patch text/x-diff 7.4 KB
v15-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch text/x-diff 9.4 KB
v15-0007-Add-pg_ls_dir_recurse-to-show-dir-recursively.patch text/x-diff 6.1 KB
v15-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch text/x-diff 3.1 KB
v15-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch text/x-diff 17.2 KB
v15-0010-pg_ls_-to-show-file-type-and-show-special-files.patch text/x-diff 17.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dean Rasheed 2020-03-31 20:23:35 Re: Berserk Autovacuum (let's save next Mandrill)
Previous Message Andres Freund 2020-03-31 20:04:38 Re: Improving connection scalability: GetSnapshotData()