Re: Why clearing the VM doesn't require registering vm buffer in wal record

Hi,

On 2026-03-05 15:38:24 -0500, Andres Freund wrote:
> > On Thu, 5 Mar 2026 at 21:16, Andres Freund <andres(at)anarazel(dot)de> wrote:
> > >
> > > But it does seem like it could be a problem for incremental backup /
> > > walsummarizer?
> >
> > I don't think it is, because that doesn't do calculations for non-main
> > forks, it considers those forks always changed and includes them in
> > full. Or at least, that was the response I got when I raised concerns
> > about the FSM back when the incremental backup feature was being
> > developed [0].
>
> There's explicit code for ignoring the FSM, but I don't see the same for the
> VM. And that makes sense: VM changes are mostly WAL logged, just not
> completely / generically (i.e. this complaint), whereas FSM changes are not
> WAL logged at all.

Unfortunately I can confirm that incremental backups end up with an outdated
VM.

An unfortunate kicker: It looks like verify_heapam() doesn't even notice :(.

But the next vacuum does complain, if we end up processing the page for some
reason:

2026-03-05 15:56:23.018 EST [2073843][client backend][0/5:0][psql] WARNING: page is not marked all-visible but visibility map bit is set in relation "s" page 0
2026-03-05 15:56:23.018 EST [2073843][client backend][0/5:0][psql] CONTEXT: while scanning block 0 of relation "public.s"

Repro:

1) CREATE TABLE s(id int); INSERT INTO s DEFAULT VALUES; SELECT pg_current_wal_insert_lsn(); VACUUM s;
2) create full base backup
rm -rf /tmp/backup-a && mkdir /tmp/backup-a && ./src/bin/pg_basebackup/pg_basebackup -c fast -D /tmp/backup-a

3) UPDATE s SET id = -id;
4) create incremental backup
rm -rf /tmp/backup-a-i && mkdir /tmp/backup-a-i && ./src/bin/pg_basebackup/pg_basebackup --incremental=/tmp/backup-a/backup_manifest -c fast -D /tmp/backup-a-i

5) combine full and incremental backup
rm -rf /tmp/backup-a-i-c && mkdir /tmp/backup-a-i-c && ./src/bin/pg_combinebackup/pg_combinebackup -d -o /tmp/backup-a-i-c --copy-file-range /tmp/backup-a /tmp/backup-a-i/

6) start PG on combined backup and use
SELECT * FROM pg_visibility_map_summary('s');
to see things are wrong

7) vacuum and see it complain (page is processed because we always look at the
last page, and there's just one)

Greetings,

Andres Freund