Re: Patch: dumping tables data in multiple chunks in pg_dump

From: David Rowley <dgrowleyml(at)gmail(dot)com>
To: Hannu Krosing <hannuk(at)google(dot)com>
Cc: Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Nathan Bossart <nathandbossart(at)gmail(dot)com>
Subject: Re: Patch: dumping tables data in multiple chunks in pg_dump
Date: 2026-01-20 02:20:13
Message-ID: CAApHDvruGg8u9t2OQbVU+Sxe31H0CuJi5S69FjZXds1Y3Fx3HA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, 20 Jan 2026 at 08:01, Hannu Krosing <hannuk(at)google(dot)com> wrote:
> I changed the last open-ended chunk to use ctid >= (N,1) for clarity
> but did not change anything else.

You have:

int max_table_segment_pages; /* chunk when relpages is above this */

and:

opts->max_table_segment_pages = UINT32_MAX; /* == InvalidBlockNumber,
disable chunking by default */

It's not valid to assign UINT32_MAX to a signed int.

> To me it looked like having a loop around the whole thing when there
> is no chunking would complicate things for anyone reading the code.

The problem I have with it is the duplicate code. If you don't want to
loop around the standard code, then make a function and call that
instead of copying and pasting the code.

I'd also get rid of the "chunking" boolean and make use of
InvalidBlockNumber to determine if the range is constrained. It also
seems very strange that you opted to do that just for endPage and not
for startPage.

> > 4. I think using "int" here is a future complaint waiting to happen.
> >
> > + if (!option_parse_int(optarg, "--huge-table-chunk-pages", 1, INT32_MAX,
> > + &dopt.huge_table_chunk_pages))
> >
> > I bet we'll eventually see a complaint that someone can't make the
> > segment size larger than 16TB. I think option_parse_uint32() might be
> > called for.
>
> I have not yet done anything with this yet, so the maximum chunk size
> for now is half of the maximum relpages.

OK. I can look again once all that's done.

David

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Henson Choi 2026-01-20 02:29:34 Re: Row pattern recognition
Previous Message Xuneng Zhou 2026-01-20 01:31:33 Re: Add WALRCV_CONNECTING state to walreceiver