Re: proposal: possibility to read dumped table's name from file

From: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
To: Justin Pryzby <pryzby(at)telsasoft(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>, Stephen Frost <sfrost(at)snowman(dot)net>, Surafel Temesgen <surafel3000(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Daniel Gustafsson <daniel(at)yesql(dot)se>
Subject: Re: proposal: possibility to read dumped table's name from file
Date: 2020-11-29 14:09:48
Message-ID: CAFj8pRCrbDPAWyHjwZauEX78jyGBowOoiaWySA0ibRYoWzZTMw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi

ne 29. 11. 2020 v 0:49 odesílatel Justin Pryzby <pryzby(at)telsasoft(dot)com>
napsal:

> On Sat, Nov 28, 2020 at 09:14:35PM +0100, Pavel Stehule wrote:
> > Any short or long option can be read from this file in simple format -
> one
> > option per line. Arguments inside double quotes can be multi lined. Row
> > comments started by # and can be used everywhere.
>

here is updated patch

> Does this support even funkier table names ?
>
> This tests a large number and fraction of characters in dbname/username,
> so all
> of pg_dump has to continue supporting that:
> ./src/bin/pg_dump/t/010_dump_connstr.pl
>
> I tested and it seems to work with -t "fooå"
> But it didn't work with -t "foo\nbar" (literal newline). Fix attached.
> If you send another patch, please consider including a test case for quoted
> names in long and short options.
>

I implemented some basic backslash escaping. I will write more tests, when
there will be good agreement on the main concept.

>
> > +static char *optsfilename = NULL;
>
> > + * It assign the values of options to related DumpOption fields or to
> > + * some global values. It is called from twice. First, for processing
> > + * the command line argumens. Second, for processing an options from
> > + * options file.
>
> This didn't support multiple config files, nor config files which include
> config files, as Dean and I mentioned. I think the argument parsers should
> themselves call the config file parser, as need be, so the last option
> specification should override previous ones.
>
> For example pg_dump --config-file=./pg_dump.conf --blobs should have blobs
> even
> if the config file says --no-blobs. (Command-line arguments normally take
> precedence over config files, certainly if the argument is specified
> "later").
> I think it'd be ok if it's recursive. I made a quick hack to do that.
>

I did it. I used a different design than you. Making "dopt" be a global
variable looks too invasive. Almost all functions there expect "dopt" as an
argument. But I think it is not necessary.

I implemented two iterations of argument's processing. 1. for options file
(more options-file options are allowed, and nesting is allowed too), 2. all
other arguments from the command line. Any options file is processed only
once - second processing is ignored. So there is no problem with cycles.

The name of the new option - "config-file" or "options-file" ? I prefer
"options-file". "config-file" is valid too, but "options-file" is more
specific, more descriptive (it is self descriptive).

I merged your patch with a fix of typos.

Regards

Pavel

> I doubt this will satisfy Stephen. Personally, I would use this if it
> were a
> plain and simple text config file (which for our purposes I would pass on
> stdin), and I would almost certainly not use it if it were json. But it'd
> be
> swell if there were a standard config file format, that handled
> postgresql.conf
> and maybe pg_hba.conf.
>
> --
> Justin
>

Attachment Content-Type Size
pg_dump-options-file-2.patch text/x-patch 30.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message James Coleman 2020-11-29 14:26:12 Re: Fix generate_useful_gather_paths for parallel unsafe pathkeys
Previous Message Kasahara Tatsuhito 2020-11-29 13:34:16 Re: autovac issue with large number of tables