| From: | Rajesh Kumar Thandapani <rajeshkumarth(at)google(dot)com> |
|---|---|
| To: | Raja Sai pranav <rajasaipranav0(at)gmail(dot)com> |
| Cc: | masao(dot)fujii(at)gmail(dot)com, pgsql-hackers(at)lists(dot)postgresql(dot)org, kehan5800(at)gmail(dot)com, pgsql-bugs(at)lists(dot)postgresql(dot)org, robertmhaas(at)gmail(dot)com, dilipkumarb(at)google(dot)com |
| Subject: | Re: Re: Re: BUG #19776: pg_combinebackup ignores the truncation_block_length of an intermediate incremental backup |
| Date: | 2026-11-16 13:36:18 |
| Message-ID: | CAJoZ81upDvURTFCt2q0O7FD9AO9EeX0eDTA+5EvwuSifWh7NJQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-bugs pgsql-hackers |
Hi Fujii-san, Sai Pranav,
Thank you both for checking that. I agree that relying on COPY
batching and shared_buffers
makes the test too sensitive to internal heuristics, so I have removed the test
change in the attached v4 patch.
One alternative I was wondering about: in 011_ib_truncation.pl, after taking
the first incremental backup (where table t is truncated from 8 blocks
to 7 blocks),
we could append one block of zeroes directly to the relation file on disk from
Perl before taking a second incremental backup:
my $relpath =
$primary->safe_psql('postgres', "SELECT pg_relation_filepath('t');");
my $segfile = $primary->data_dir . "/$relpath";
$segfile .= ".1" if -f "$segfile.1";
open(my $fh, '>>:raw', $segfile) or die "open $segfile: $!";
print $fh ("\0" x $block_size);
close($fh);
That simulates an unwritten zero-filled page at block 7 without relying on COPY
or shared_buffers, and deterministically reproduces the bug (returning 8 rows
instead of 7 without the fix).
Do you think a test like that is worth adding, or is it better to leave out the
test and go with v4 as-is?
I am okay with either option; I'm just sharing my thoughts on adding a test
without assuming any internal huristics.
Thanks.
Regards,
Rajesh Kumar
On Mon, Nov 16, 2026 at 11:33 AM Raja Sai pranav <rajasaipranav0(at)gmail(dot)com>
wrote:
> Hi Fujii-san,
>
> I checked this by running
> https://www.google.com/url?q=http://011_ib_truncation.pl&source=gmail&ust=1794895380937000&sa=E
> against current master without the reconstruct.c change, varying $copy_rows
> (1, 4, 16, 17, 18, 32, 100) and shared_buffers (8MB, 16MB, 128MB). The new
> subtest catches the bug in only 6 of the 21 combinations (copy_rows 16-17
> with 8MB, 17-18 with 16MB and 128MB); in all the others it passes without
> the fix. So I agree it depends too much on COPY's batching to be a reliable
> regression test, and dropping it makes sense, since the reproducer upthread
> already demonstrates the bug and the fix.
>
> Regards,
> Sai Pranav
>
> On Mon, Nov 16, 2026 10:42 AM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>
>> On Sun, Nov 8, 2026 at 12:21 AM Raja Sai pranav
>> <rajasaipranav0(at)gmail(dot)com> wrote:
>> > - v3 now adds the bulk-extension regression test I'd flagged as
>> missing: the
>> > node3 / incr2-incr4 chain that bumps shared_buffers so COPY
>> bulk-extends the
>> > relation past an intermediate truncation point, leaving zero-filled
>> unwritten
>> > pages in the truncated range.
>>
>> I just wonder if the added test might be fragile. It relies on internal
>> logics, such as COPY's batching behavior ($copy_rows = 17), buffer pin
>> limits (shared_buffers = 8MB), and the bulk-extension strategy leaving
>> pages unwritten. Changes to any of these could easily make the test pass
>> even without the fix. No?
>>
>> If so, isn't it better to leave out the test for now? Also because
>> the reproducer has already demonstrated the bug and the fix.
>>
>> Regards,
>>
>>
>> --
>> Fujii Masao
>>
>
| Attachment | Content-Type | Size |
|---|---|---|
| v4-0001-Fix-pg_combinebackup-with-truncated-intermediate-.patch | application/x-patch | 3.8 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Joao Rito | 2026-11-16 14:02:39 | Re: BUG #19817: CREATE UNIQUE INDEX on a partitioned table can silently attach an invalid, not-ready child index, le |
| Previous Message | Hayato Kuroda (Fujitsu) | 2026-11-16 13:08:12 | RE: BUG #19892: Logical replication skips UPDATE/DELETE of rows with NULL in a hash-indexed subscriber column |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tomas Vondra | 2026-11-16 13:38:40 | contain_nonstrict_functions_walker() fails to consider JsonExpr and CoerceToDomain |
| Previous Message | Tomás Senart | 2026-11-16 13:00:33 | [PATCH v1] Avoid deforming unused trailing columns in COPY TO |