| From: | Melanie Plageman <melanieplageman(at)gmail(dot)com> |
|---|---|
| To: | Kirill Reshke <reshkekirill(at)gmail(dot)com> |
| Cc: | Andres Freund <andres(at)anarazel(dot)de>, Robert Haas <robertmhaas(at)gmail(dot)com>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> |
| Subject: | Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access) |
| Date: | 2025-11-17 23:07:27 |
| Message-ID: | CAAKRu_ZGZSqhGt-RcmmfiSheC+1fjQdxy6_+oM-1jMn8hyVptQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Attached v20 has general cleanup, changes to the table/index AM
callbacks detailed below, and it moves the
heap_page_prune_and_freeze() refactoring commit down the stack to
0004.
0001 - 0003 are fairly trivial cleanup patches. I think they are ready
to commit, so if I don't hear any objections in the next few days,
I'll go ahead and commit them.
On Tue, Nov 4, 2025 at 11:48 AM Melanie Plageman
<melanieplageman(at)gmail(dot)com> wrote:
>
> On Wed, Oct 29, 2025 at 7:03 AM Kirill Reshke <reshkekirill(at)gmail(dot)com> wrote:
> >
> > In 0010:
> >
> > I'm not terribly convenient that adding SO_ALLOW_VM_SET to TAM
> > ScanOptions is the right thing to do. Looks like VM bits are something
> > that make sense for HEAP AM for not for any TAM. So, don't we break
> > some layer of abstraction here? Would it be better for HEAP AM to set
> > some flags in heap_beginscan?
>
> I don't see another good way of doing it.
>
> The information about whether or not the relation is modified in the
> query is gathered during planning and saved in the plan. We need to
> get that information to the scan descriptor, which is all we have when
> we call heap_page_prune_opt() during the scan. The scan descriptor is
> created by the table AM implementations of scan_begin(). The table AM
> callbacks don't pass down the plan -- which makes sense; the scan
> shouldn't know about the plan. They do pass down flags, so I thought
> it made the most sense to add a flag. Note that I was able to avoid
> modifying the actual table and index AM callbacks (scan_begin() and
> ambeginscan()). I only made new wrappers that took "modifies_rel".
>
> Now, it is true that referring to the VM is somewhat of a layering
> violation. Though, other table AMs may use the information about if
> the query modifies the relation -- which is really what this flag
> represents. The ScanOptions are usually either a type or a call to
> action. Which is why I felt a bit uncomfortable calling it something
> like SO_MODIFIES_REL -- which is less of an option and more a piece of
> information. And it makes it sound like the scan modifies the rel,
> which is not the case. I wonder if there is another solution. Or maybe
> we call it SO_QUERY_MODIFIES_REL?
Attached v20 changes the ScanOption name to SO_HINT_REL_READ_ONLY and
removes the new helper functions which took modifies_rel as a
parameter. Instead it modifies the existing
table_beginscan()/index_beginscan() helpers and the relevant callbacks
they invoke to have a new flags parameter. These are additional caller
provider flags.
In master, the IndexScan structures and helpers don't use ScanOptions,
but since I'm using them for properties of the base relation, I think
it is fine. I'm not sure if I should name the parameter base_rel_flags
instead of flags for the index-related callbacks and helpers or if
leaving it more generic is better, though.
- Melanie
| Attachment | Content-Type | Size |
|---|---|---|
| v20-0001-Refactor-heap_page_prune_and_freeze-parameters-i.patch | text/x-patch | 13.0 KB |
| v20-0002-Keep-all_frozen-updated-in-heap_page_prune_and_f.patch | text/x-patch | 9.1 KB |
| v20-0003-Update-PruneState.all_-visible-frozen-earlier-in.patch | text/x-patch | 9.7 KB |
| v20-0004-Split-heap_page_prune_and_freeze-into-helpers.patch | text/x-patch | 25.7 KB |
| v20-0005-Eliminate-XLOG_HEAP2_VISIBLE-from-vacuum-phase-I.patch | text/x-patch | 43.5 KB |
| v20-0006-Eliminate-XLOG_HEAP2_VISIBLE-from-empty-page-vac.patch | text/x-patch | 2.5 KB |
| v20-0007-Remove-XLOG_HEAP2_VISIBLE-entirely.patch | text/x-patch | 26.2 KB |
| v20-0008-Rename-GlobalVisTestIsRemovableXid-to-GlobalVisX.patch | text/x-patch | 8.0 KB |
| v20-0009-Use-GlobalVisState-in-vacuum-to-determine-page-l.patch | text/x-patch | 10.5 KB |
| v20-0010-Unset-all_visible-sooner-if-not-freezing.patch | text/x-patch | 2.4 KB |
| v20-0011-Allow-on-access-pruning-to-set-pages-all-visible.patch | text/x-patch | 37.7 KB |
| v20-0012-Set-pd_prune_xid-on-insert.patch | text/x-patch | 6.7 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Michael Paquier | 2025-11-17 23:19:58 | Re: Extended Statistics set/restore/clear functions. |
| Previous Message | Shinya Kato | 2025-11-17 22:52:58 | Re: Speed up COPY FROM text/CSV parsing using SIMD |