| From: | Raja Sai pranav <rajasaipranav0(at)gmail(dot)com> |
|---|---|
| To: | rajeshkumarth(at)google(dot)com |
| Cc: | masao(dot)fujii(at)gmail(dot)com, pgsql-hackers(at)lists(dot)postgresql(dot)org, kehan5800(at)gmail(dot)com, pgsql-bugs(at)lists(dot)postgresql(dot)org, robertmhaas(at)gmail(dot)com, dilipkumarb(at)google(dot)com |
| Subject: | Re: BUG #19776: pg_combinebackup ignores the truncation_block_length of an intermediate incremental backup |
| Date: | 2026-11-17 04:17:06 |
| Message-ID: | CAPHduZBhjjtu1ewFXoN1ZEzhKHQFVPfmrkwxbNPfdni9Z2PXKQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-bugs pgsql-hackers |
Hi Rajesh,
I tried the zero-block variant on current master (5ffc821037) with v4's
reconstruct.c, adding the append plus a second incremental backup and a
third node to
https://www.google.com/url?q=http://011_ib_truncation.pl&source=gmail&ust=1794975425465000&sa=E,
and ran it with shared_buffers at 8MB and 128MB. With the fix it passes at
both sizes; without the fix it fails at both with 8 rows instead of 7. So
unlike the v3 subtest it reproduces the bug deterministically, and nothing
in it depends on COPY batching.
I think it is worth adding. The only thing I would add is a comment saying
why a zero page is appended to the heap file directly, since that is the
exact shape the bug mishandles (an unwritten block past the truncation
point).
Regards,
Sai Pranav
On Mon, Nov 16, 2026 07:06 PM, Rajesh Kumar Thandapani <
rajeshkumarth(at)google(dot)com> wrote:
> Hi Fujii-san, Sai Pranav,
>
> Thank you both for checking that. I agree that relying on COPY batching and
> shared_buffers makes the test too sensitive to internal heuristics, so I have
> removed the test change in the attached v4 patch.
>
> One alternative I was wondering about: in 011_ib_truncation.pl, after taking
> the first incremental backup (where table t is truncated from 8 blocks to 7
> blocks), we could append one block of zeroes directly to the relation file
> on disk from Perl before taking a second incremental backup:
>
> my $relpath =
> $primary->safe_psql('postgres', "SELECT pg_relation_filepath('t');");
> my $segfile = $primary->data_dir . "/$relpath";
> $segfile .= ".1" if -f "$segfile.1";
> open(my $fh, '>>:raw', $segfile) or die "open $segfile: $!";
> print $fh ("\0" x $block_size);
> close($fh);
>
> That simulates an unwritten zero-filled page at block 7 without relying on
> COPY or shared_buffers, and deterministically reproduces the bug (returning
> 8 rows instead of 7 without the fix).
>
> Do you think a test like that is worth adding, or is it better to leave out
> the test and go with v4 as-is?
> I am okay with either option; I'm just sharing my thoughts on adding a
> test without assuming any internal huristics.
>
> Thanks.
>
> Regards,
> Rajesh Kumar
>
> On Mon, Nov 16, 2026 at 11:33 AM Raja Sai pranav <rajasaipranav0(at)gmail(dot)com>
> wrote:
>
>> Hi Fujii-san,
>>
>> I checked this by running https://www.google.com/url?q=
>> http://011_ib_truncation.pl&source=gmail&ust=1794895380937000&sa=E
>> against current master without the reconstruct.c change, varying $copy_rows
>> (1, 4, 16, 17, 18, 32, 100) and shared_buffers (8MB, 16MB, 128MB). The new
>> subtest catches the bug in only 6 of the 21 combinations (copy_rows 16-17
>> with 8MB, 17-18 with 16MB and 128MB); in all the others it passes without
>> the fix. So I agree it depends too much on COPY's batching to be a reliable
>> regression test, and dropping it makes sense, since the reproducer upthread
>> already demonstrates the bug and the fix.
>>
>> Regards,
>> Sai Pranav
>>
>> On Mon, Nov 16, 2026 10:42 AM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>>
>>> On Sun, Nov 8, 2026 at 12:21 AM Raja Sai pranav
>>> <rajasaipranav0(at)gmail(dot)com> wrote:
>>> > - v3 now adds the bulk-extension regression test I'd flagged as
>>> missing: the
>>> > node3 / incr2-incr4 chain that bumps shared_buffers so COPY
>>> bulk-extends the
>>> > relation past an intermediate truncation point, leaving zero-filled
>>> unwritten
>>> > pages in the truncated range.
>>>
>>> I just wonder if the added test might be fragile. It relies on internal
>>> logics, such as COPY's batching behavior ($copy_rows = 17), buffer pin
>>> limits (shared_buffers = 8MB), and the bulk-extension strategy leaving
>>> pages unwritten. Changes to any of these could easily make the test pass
>>> even without the fix. No?
>>>
>>> If so, isn't it better to leave out the test for now? Also because
>>> the reproducer has already demonstrated the bug and the fix.
>>>
>>> Regards,
>>>
>>>
>>> --
>>> Fujii Masao
>>>
>>
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Adam Matthews | 2026-11-17 04:21:55 | Re: 16.16: join order regression for pg_indexes self-join (schema filter applied last), ~50x slower than 16.15 |
| Previous Message | David Rowley | 2026-11-17 03:55:15 | Re: BUG #19904: pg_column_is_updatable(): "negative bitmapset member not allowed" for large attnum (int16 overflow) |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Sami Imseih | 2026-11-17 04:39:26 | Re: Improve pg_stat_statements scalability |
| Previous Message | Chee Wooson | 2026-11-17 03:20:52 | Re: [PATCH v2] Avoid carrying self lock-only xmax to updated tuple |