Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)

From: Melanie Plageman <melanieplageman(at)gmail(dot)com>
To: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
Cc: Kirill Reshke <reshkekirill(at)gmail(dot)com>, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>, Xuneng Zhou <xunengzhou(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Subject: Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)
Date: 2026-01-06 17:31:57
Message-ID: CAAKRu_YQd=2KvomM+RHcpeDKj0bq+peJ=3W-fip+pkvzA-Jq9w@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Jan 6, 2026 at 4:40 AM Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
>
> > <v32-0014-Pass-down-information-on-table-modification-to-s.patch>
>
> I've tried to take an attempt to review some patches of this patchset. It's huge and mostly polished.

I've added attributed your review on the patches you specifically
mention here (and from previous emails you sent). Let me know if there
are other patches you reviewed that you did not mention.

> In a step "Pass down information on table modification to scan node" you pass SO_HINT_REL_READ_ONLY flag in IndexNext() and BitmapTableScanSetup(), but not in IndexNextWithReorder() and IndexOnlyNext(). Is there a reason why index scans with ordering cannot use on-access VM setting?

Great point, I simply hadn't tested those cases and didn't think to
add them. I've added them in attached v33.

While looking at other callers of index_beginscan(), I was wondering
if systable_beginscan() and systable_beginscan_ordered() should ever
pass SO_HINT_REL_READ_ONLY. I guess we would need to pass if the
operation is read-only above the index_beginscan() -- I'm not sure if
we always know in the caller of systable_beginscan() whether this
operation will modify the catalog. That seems like it could be a
separate project, though, so maybe it is better to say this feature is
just for regular tables.

As for the other cases: We don't have the relation range table index
in check_exclusion_or_unique_constraints(), so I don't think we can do
it there.

And I think that the other index scan cases like in replication code
or get_actual_variable_endpoint() are too small to be worth it, don't
have the needed info, or don't do on-access pruning (bc of the
snapshot type they use).

> Also, comment about visibilitymap_set() says "Callers that log VM changes separately should use visibilitymap_set()" as if visibilitymap_set() is some other function.

Ah, yes, I forgot to remove that when I removed the old
visibilitymap_set() and made visibilitymap_set_vmbits() into
visiblitymap_set(). Done in v33.

- Melanie

Attachment Content-Type Size
v33-0001-Combine-visibilitymap_set-cases-in-lazy_scan_pru.patch text/x-patch 10.2 KB
v33-0002-Eliminate-use-of-cached-VM-value-in-lazy_scan_pr.patch text/x-patch 16.9 KB
v33-0003-Refactor-lazy_scan_prune-VM-clear-logic-into-hel.patch text/x-patch 6.7 KB
v33-0004-Set-the-VM-in-heap_page_prune_and_freeze.patch text/x-patch 26.8 KB
v33-0005-Move-VM-assert-into-prune-freeze-code.patch text/x-patch 10.9 KB
v33-0006-Eliminate-XLOG_HEAP2_VISIBLE-from-vacuum-phase-I.patch text/x-patch 14.2 KB
v33-0007-Eliminate-XLOG_HEAP2_VISIBLE-from-empty-page-vac.patch text/x-patch 2.6 KB
v33-0008-Remove-XLOG_HEAP2_VISIBLE-entirely.patch text/x-patch 24.9 KB
v33-0009-Simplify-heap_page_would_be_all_visible-visibili.patch text/x-patch 2.4 KB
v33-0010-Remove-table_scan_analyze_next_tuple-unneeded-pa.patch text/x-patch 4.7 KB
v33-0011-Use-GlobalVisState-in-vacuum-to-determine-page-l.patch text/x-patch 11.9 KB
v33-0012-Unset-all_visible-sooner-if-not-freezing.patch text/x-patch 2.5 KB
v33-0013-Track-which-relations-are-modified-by-a-query.patch text/x-patch 2.6 KB
v33-0014-Pass-down-information-on-table-modification-to-s.patch text/x-patch 24.7 KB
v33-0015-Allow-on-access-pruning-to-set-pages-all-visible.patch text/x-patch 11.0 KB
v33-0016-Set-pd_prune_xid-on-insert.patch text/x-patch 6.8 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Marcos Magueta 2026-01-06 18:03:15 Re: WIP - xmlvalidate implementation from TODO list
Previous Message Xuneng Zhou 2026-01-06 17:04:06 Re: Implement waiting for wal lsn replay: reloaded