| From: | Melanie Plageman <melanieplageman(at)gmail(dot)com> |
|---|---|
| To: | Andres Freund <andres(at)anarazel(dot)de> |
| Cc: | Kirill Reshke <reshkekirill(at)gmail(dot)com>, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, Xuneng Zhou <xunengzhou(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> |
| Subject: | Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access) |
| Date: | 2026-03-23 21:54:13 |
| Message-ID: | CAAKRu_bgP-DMZs=D2j2N0+U9+uWU5cVagw-yZLOuhYbWj_KwnA@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Sun, Mar 22, 2026 at 3:58 PM Melanie Plageman
<melanieplageman(at)gmail(dot)com> wrote:
>
> I've pushed the first two patches. Attached are the remaining 10. No
> changes were made to those from the previous version.
I'm planning on pushing 0001-0005 in the morning.
I've made some significant changes to 0006 and realized I need some
help. 0006 tracks what relations are modified by a query. This new
version (v44) uses relation oids instead of rt indexes to handle cases
where the same relation appears more than once in the range table
(e.g. INSERT INTO foo SELECT * FROM foo; foo appears twice). It
computes modifiedRelOids (a list of relation OIDs modified by the
query) in the planner and stores them in the PlannedStmt. There is one
big issue I'm not sure how to solve:
For queries like INSERT INTO ptable SELECT * FROM ptable, where ptable
is a partitioned table, though we scan ptable, we don't know when
executing that scan that we will then modify ptable with the insert.
In my patch, I've added find_all_inheritors() when populating
modifiedRelOids, but I realize this probably isn't acceptable to add
to planner from a performance perspective.
I'm looking for other ways to solve the problem. Now, for my use case
(setting the VM), we don't mind setting the VM during the table scan
part of the query. Whatever page gets the inserted tuple will clear
all-visible -- but that is just one page out of many. However, future
users of modifiedRelOids will likely expect it to contain all modified
relation oids.
I could also check when setting up the scan descriptor if the leaf
partition's parents (would have to check full ancestry) are in
modifiedRelOids. This also doesn't address the problem of future users
thinking modifiedRelOids is complete.
Note that it also means partitions that aren't modified will be
included in modifiedRelOids if one of the partitions is being
modified.
I could also just change the name of the modifiedRelOids to something
that doesn't make future users think it's exhaustive.
- Melanie
| Attachment | Content-Type | Size |
|---|---|---|
| v44-0001-Use-GlobalVisState-in-vacuum-to-determine-page-l.patch | text/x-patch | 18.4 KB |
| v44-0002-Keep-newest-live-XID-up-to-date-even-if-page-not.patch | text/x-patch | 15.6 KB |
| v44-0003-WAL-log-VM-setting-during-vacuum-phase-I-in-XLOG.patch | text/x-patch | 23.2 KB |
| v44-0004-WAL-log-VM-setting-for-empty-pages-in-XLOG_HEAP2.patch | text/x-patch | 5.7 KB |
| v44-0005-Remove-XLOG_HEAP2_VISIBLE-entirely.patch | text/x-patch | 27.6 KB |
| v44-0006-Track-which-relations-are-modified-by-a-query.patch | text/x-patch | 6.4 KB |
| v44-0007-Thread-flags-through-begin-scan-APIs.patch | text/x-patch | 32.9 KB |
| v44-0008-Pass-down-information-on-table-modification-to-s.patch | text/x-patch | 11.8 KB |
| v44-0009-Allow-on-access-pruning-to-set-pages-all-visible.patch | text/x-patch | 10.1 KB |
| v44-0010-Set-pd_prune_xid-on-insert.patch | text/x-patch | 8.8 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Mihail Nikalayeu | 2026-03-23 22:08:00 | Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements |
| Previous Message | Zsolt Parragi | 2026-03-23 21:45:38 | Re: Custom oauth validator options |