Re: Patch: dumping tables data in multiple chunks in pg_dump

From: David Rowley <dgrowleyml(at)gmail(dot)com>
To: Hannu Krosing <hannuk(at)google(dot)com>
Cc: Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Nathan Bossart <nathandbossart(at)gmail(dot)com>
Subject: Re: Patch: dumping tables data in multiple chunks in pg_dump
Date: 2026-01-14 21:10:55
Message-ID: CAApHDvr1nRk=ZDszZOUrh-UXbE-6Fi6cDN5pEF+ZEcqJRnZbfw@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, 14 Jan 2026 at 23:52, Hannu Krosing <hannuk(at)google(dot)com> wrote:
>
> On Tue, Jan 13, 2026 at 3:27 AM David Rowley <dgrowleyml(at)gmail(dot)com> wrote:
> > On testing Citus's columnar AM, I get:
> > postgres=# select * from t where ctid between '(0,1)' and '(10,0)';
> > ERROR: UPDATE and CTID scans not supported for ColumnarScan
>
> Should we just silently not chunk tables that have some storage
> architecture that does not have tids, or should pg_dump just error out
> in thiscase ?

I think you should just document that it only applies to heap tables.
I don't think erroring out is useful to anyone, especially if the
error only arrives after pg_dump has been running for several hours or
even days.

> > 4. I think using "int" here is a future complaint waiting to happen.
> >
> > + if (!option_parse_int(optarg, "--huge-table-chunk-pages", 1, INT32_MAX,
> > + &dopt.huge_table_chunk_pages))
> >
> > I bet we'll eventually see a complaint that someone can't make the
> > segment size larger than 16TB. I think option_parse_uint32() might be
> > called for.
>
> There can be no more than 2 * INT2_MAX pages anyway.
> I thought half of the max possible size should be enough.
> Do you really think that somebody would want that ?

IMO, if the option can't represent the full range of BlockNumber, then
that's a bug.

I see pg_resetwal has recently invented strtouint32_strict for this.
It might be a good idea to refactor that and put it into
option_utils.c rather than having each client app have to invent their
own method.

David

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2026-01-14 21:12:04 Re: Updating IPC::Run in CI?
Previous Message Kirill Reshke 2026-01-14 21:03:07 Re: PL/Python initialization cleanup