Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)

From: Melanie Plageman <melanieplageman(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Kirill Reshke <reshkekirill(at)gmail(dot)com>, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, Xuneng Zhou <xunengzhou(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Subject: Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)
Date: 2026-03-23 21:54:13
Message-ID: CAAKRu_bgP-DMZs=D2j2N0+U9+uWU5cVagw-yZLOuhYbWj_KwnA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sun, Mar 22, 2026 at 3:58 PM Melanie Plageman
<melanieplageman(at)gmail(dot)com> wrote:
>
> I've pushed the first two patches. Attached are the remaining 10. No
> changes were made to those from the previous version.

I'm planning on pushing 0001-0005 in the morning.

I've made some significant changes to 0006 and realized I need some
help. 0006 tracks what relations are modified by a query. This new
version (v44) uses relation oids instead of rt indexes to handle cases
where the same relation appears more than once in the range table
(e.g. INSERT INTO foo SELECT * FROM foo; foo appears twice). It
computes modifiedRelOids (a list of relation OIDs modified by the
query) in the planner and stores them in the PlannedStmt. There is one
big issue I'm not sure how to solve:

For queries like INSERT INTO ptable SELECT * FROM ptable, where ptable
is a partitioned table, though we scan ptable, we don't know when
executing that scan that we will then modify ptable with the insert.

In my patch, I've added find_all_inheritors() when populating
modifiedRelOids, but I realize this probably isn't acceptable to add
to planner from a performance perspective.

I'm looking for other ways to solve the problem. Now, for my use case
(setting the VM), we don't mind setting the VM during the table scan
part of the query. Whatever page gets the inserted tuple will clear
all-visible -- but that is just one page out of many. However, future
users of modifiedRelOids will likely expect it to contain all modified
relation oids.

I could also check when setting up the scan descriptor if the leaf
partition's parents (would have to check full ancestry) are in
modifiedRelOids. This also doesn't address the problem of future users
thinking modifiedRelOids is complete.

Note that it also means partitions that aren't modified will be
included in modifiedRelOids if one of the partitions is being
modified.

I could also just change the name of the modifiedRelOids to something
that doesn't make future users think it's exhaustive.

- Melanie

Attachment Content-Type Size
v44-0001-Use-GlobalVisState-in-vacuum-to-determine-page-l.patch text/x-patch 18.4 KB
v44-0002-Keep-newest-live-XID-up-to-date-even-if-page-not.patch text/x-patch 15.6 KB
v44-0003-WAL-log-VM-setting-during-vacuum-phase-I-in-XLOG.patch text/x-patch 23.2 KB
v44-0004-WAL-log-VM-setting-for-empty-pages-in-XLOG_HEAP2.patch text/x-patch 5.7 KB
v44-0005-Remove-XLOG_HEAP2_VISIBLE-entirely.patch text/x-patch 27.6 KB
v44-0006-Track-which-relations-are-modified-by-a-query.patch text/x-patch 6.4 KB
v44-0007-Thread-flags-through-begin-scan-APIs.patch text/x-patch 32.9 KB
v44-0008-Pass-down-information-on-table-modification-to-s.patch text/x-patch 11.8 KB
v44-0009-Allow-on-access-pruning-to-set-pages-all-visible.patch text/x-patch 10.1 KB
v44-0010-Set-pd_prune_xid-on-insert.patch text/x-patch 8.8 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Mihail Nikalayeu 2026-03-23 22:08:00 Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements
Previous Message Zsolt Parragi 2026-03-23 21:45:38 Re: Custom oauth validator options