Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)

From: Melanie Plageman <melanieplageman(at)gmail(dot)com>
To: Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>
Cc: Andres Freund <andres(at)anarazel(dot)de>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, Kirill Reshke <reshkekirill(at)gmail(dot)com>, Xuneng Zhou <xunengzhou(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Subject: Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)
Date: 2026-03-11 17:01:36
Message-ID: CAAKRu_Z8Ry_ynNBPAzs_Ry3MQi9NaBgt1ccLgwRsDbxWpocaBg@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Mar 6, 2026 at 6:33 PM Melanie Plageman
<melanieplageman(at)gmail(dot)com> wrote:
>
> Thanks for the review! Attached is v36. I've pushed some of the early
> patches in the set and this is what is left.

I've gone ahead and pushed another of the introductory commits.
Attached v37 has the remaining patches.

The one change is that I've removed get_conflict_xid(). I determined
that in the current code that we cannot end up in the scenario where
we didn't prune or freeze and the page was already all-visible but not
all-frozen. The closest scenario would be one where the page was
all-frozen, we cleared the all-frozen bit because we did a SELECT FOR
UPDATE on one of the tuples, then vacuum freezes the page. Even though
we are just invalidating the xmax, it still counts as freezing.
However, in this case we will not advance the FreezePageConflictXid,
so the snapshot conflict horizon will still correctly be
InvalidTransactionId. I believe in all cases we will correctly set the
conflict horizon to InvalidTransactionId with this much simpler
conflict xid calculation:

conflict_xid = InvalidTransactionId;
if (do_set_vm)
conflict_xid = prstate.newest_live_xid;
if (do_freeze &&
TransactionIdFollows(prstate.pagefrz.FreezePageConflictXid,
conflict_xid))
conflict_xid = prstate.pagefrz.FreezePageConflictXid;
if (do_prune && TransactionIdFollows(prstate.latest_xid_removed,
conflict_xid))
conflict_xid = prstate.latest_xid_removed;

The only outstanding question I have is about pd_prune_xid on insert:

I think we only want to set pd_prune_xid on insert if the transaction
ID is normal. Bootstrap mode does call heap_insert(), so we need to
check the xid before setting it. The only question is then if we want
the same guard on replay. Bootstrap mode won't actually insert a WAL
record, so we don't need this check I think. However, I think it is
better to have it for consistency with normal mode.

- Melanie

Attachment Content-Type Size
v37-0001-Save-vmbuffer-in-heap-specific-scan-descriptors-.patch text/x-patch 6.2 KB
v37-0002-Fix-visibility-map-corruption-in-more-cases.patch text/x-patch 18.6 KB
v37-0003-Add-pruning-fast-path-for-all-visible-and-all-fr.patch text/x-patch 4.5 KB
v37-0004-Use-GlobalVisState-in-vacuum-to-determine-page-l.patch text/x-patch 11.4 KB
v37-0005-Keep-newest-live-XID-up-to-date-even-if-page-not.patch text/x-patch 14.8 KB
v37-0006-Eliminate-XLOG_HEAP2_VISIBLE-from-vacuum-phase-I.patch text/x-patch 24.7 KB
v37-0007-Eliminate-XLOG_HEAP2_VISIBLE-from-empty-page-vac.patch text/x-patch 2.6 KB
v37-0008-Remove-XLOG_HEAP2_VISIBLE-entirely.patch text/x-patch 25.0 KB
v37-0009-Initialize-missing-fields-in-CreateExecutorState.patch text/x-patch 1.0 KB
v37-0010-Track-which-relations-are-modified-by-a-query.patch text/x-patch 5.5 KB
v37-0011-Thread-flags-through-begin-scan-APIs.patch text/x-patch 21.5 KB
v37-0012-Pass-down-information-on-table-modification-to-s.patch text/x-patch 8.0 KB
v37-0013-Allow-on-access-pruning-to-set-pages-all-visible.patch text/x-patch 9.8 KB
v37-0014-Avoid-BufferGetPage-calls-in-heap_update.patch text/x-patch 5.6 KB
v37-0015-Set-pd_prune_xid-on-insert.patch text/x-patch 10.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Ashutosh Bapat 2026-03-11 17:04:38 Re: SQL Property Graph Queries (SQL/PGQ)
Previous Message shihao zhong 2026-03-11 16:50:14 Re: [PATCH] Simplify ExecWithoutOverlapsNotEmpty by removing unused parameter