Re: proposal: possibility to read dumped table's name from file

From: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
To: Justin Pryzby <pryzby(at)telsasoft(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: proposal: possibility to read dumped table's name from file
Date: 2020-06-09 09:46:24
Message-ID: CAFj8pRBP105Qa=2q9MEy_PWmCZGaWd4nt4nB2Gv8WgBO7UbkwA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

po 8. 6. 2020 v 23:30 odesílatel Justin Pryzby <pryzby(at)telsasoft(dot)com>
napsal:

> On Mon, Jun 08, 2020 at 07:18:49PM +0200, Pavel Stehule wrote:
> > pá 29. 5. 2020 v 20:25 odesílatel Justin Pryzby <pryzby(at)telsasoft(dot)com>
> napsal:
> >
> > > On Fri, May 29, 2020 at 04:21:00PM +0200, Pavel Stehule wrote:
> > > > one my customer has to specify dumped tables name by name. After
> years and
> > > > increasing database size and table numbers he has problem with too
> short
> > > > command line. He need to read the list of tables from file (or from
> stdin).
> > >
> > > +1 - we would use this.
> > >
> > > We put a regex (actually a pg_dump pattern) of tables to skip
> (timeseries
> > > partitions which are older than a few days and which are also dumped
> once not
> > > expected to change, and typically not redumped). We're nowhere near
> the
> > > execve() limit, but it'd be nice if the command was primarily a list
> of options
> > > and not a long regex.
> > >
> > > Please also support reading from file for --exclude-table=pattern.
> > >
> > > I'm drawing a parallel between this and rsync --include/--exclude and
> > > --filter.
> > >
> > > We'd be implementing a new --filter, which might have similar syntax
> to rsync
> > > (which I always forget).
> >
> > I implemented support for all "repeated" pg_dump options.
> >
> > I invite any help with doc. There is just very raw text
> >
> > + Do not dump data of tables spefified in file.
>
> *specified
>
>
I am sending updated version - now with own implementation GNU (not POSIX)
function getline

I still wonder if a better syntax would use a unified --filter option, whose
> argument would allow including/excluding any type of object:
>
> +[tn] include (t)table/(n)namespace/...
> -[tn] exclude (t)table/(n)namespace/...
>
> In the past, I looked for a way to exclude extended stats objects, and
> ended up
> using a separate schema. An "extensible" syntax might be better (although
> reading a file of just patterns has the advantage that the function can
> just be
> called once for each option for each type of object).
>

I tried to implement simple format "[+-][tndf] objectname"

please, check attached patch

Regards

Pavel

>
> --
> Justin
>

Attachment Content-Type Size
pg_dump-filter.patch text/x-patch 6.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Gierth 2020-06-09 10:08:34 Re: Speedup usages of pg_*toa() functions
Previous Message Amit Kapila 2020-06-09 09:34:33 Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions