Re: WAL_LOG CREATE DATABASE strategy broken for non-standard page layouts

From: Melanie Plageman <melanieplageman(at)gmail(dot)com>
To: Zhijie Hou <houzhijie22(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, Rogers Wang <rogers(dot)ww(at)qq(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org, Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, Álvaro Herrera <alvherre(at)kurilemu(dot)de>, Andres Freund <andres(at)anarazel(dot)de>
Subject: Re: WAL_LOG CREATE DATABASE strategy broken for non-standard page layouts
Date: 2026-11-17 22:47:48
Message-ID: CAAKRu_YshjAdgXWd2_bQ-CEH8erG6dMSrsJa2gag-ueAYEij6A@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Nov 17, 2026 at 10:21 AM Zhijie Hou <houzhijie22(at)gmail(dot)com> wrote:
>
> I think we also need to increase shared_buffers (e.g., to 16MB), because
> $node->init(allows_streaming => 1) sets shared_buffers to 1MB
> internally, causing the pg_class pages to be evicted from shared_buffers
> before CREATE TABLE runs, so the test could not reach the old bug path.

Ah, okay. I'll do that. The original test feels awfully fragile. Is
there something we could do to strengthen it? Like check that one of
the buffers is marked BM_PERMANENT somehow? It seems very trivial to
accidentally eliminate the coverage. And there are of course lots of
ways for those buffers to get evicted.

> BTW, while looking into why the wal_consistency_checking test didn't
> catch this bug, I noticed that verifyBackupPageConsistency() also masks
> out all the bits in VM pages via heap_mask() (somewhat similar to this
> bug, since it treats pd_lower..pd_upper as unused space), so the
> consistency check for VM pages always "passes". Once we avoid masking
> the VM bitmap there (see the attached draft patch for reference), the
> existing test catches this CREATE DB WAL_LOG bug right away:
>
> PG_TEST_EXTRA=wal_consistency_checking meson test -C build
> recovery/027_stream_regress
>
> So if it's worth increasing the possibility of catching similar VM bugs,
> maybe we could improve that as well (in a separate patch or thread).

Ha, I didn't realize that wal_consistency_checking wasn't validating
the VM pages at all basically. I have a patch in this thread [1] to
stop masking PD_ALL_VISIBLE on heap pages, which I planned to commit
along with some other VM hardening in the next few weeks. Seems like I
should also do something about masking the whole VM pages themselves
too.

- Melanie

[1] https://www.postgresql.org/message-id/CAAKRu_aRdQ6RjneKeQh9%2BRVMFgMrtjTYFX%3DzDgBdSiQ8JDs6Jg%40mail.gmail.com

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2026-11-17 22:52:46 Re: [PATCH] Fall back to serializing when parallel apply runs out of DSM segments
Previous Message Nathan Bossart 2026-11-17 22:39:05 Re: create .abi-compliance-history file for v19