| From: | Lukas Fittl <lukas(at)fittl(dot)com> |
|---|---|
| To: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Unify parallel worker handling for index builds and instrumentation |
| Date: | 2026-05-31 19:01:40 |
| Message-ID: | CAP53Pkw3WcVbg61G29AmN=6XkuxGryoWKnhTyC1JmkT08i8qVA@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
Whilst working on the stack-based instrumentation patch, I noticed
that there is a lot of duplication of parallel worker code, and in
particular parallel index build code, when it comes to how it sets up
shared memory, and how metadata information is fed back to/from the
leader.
My initial focus was around instrumentation, but I think there is a
bigger duplication issue that needs a refactoring in regards to
parallel index builds in particular. See attached a draft patch
series, that does the following:
0001: Use the new Instrumentation struct to handle WAL and Buffer usage together
0002: New helpers to estimate/store Instrumentation struct in shared memory
0003: New helpers to handle query text for parallel workers
0004: Introduces shared parallel index build helper functions
Of these, 0004 is where I have the most uncertainty personally (what's
a good abstraction here?), but wanted to share this to spark a
conversation on how we could improve this.
I could imagine an alternate design where we refactor more generally
how parallel workers get handled for any maintenance commands (i.e.
also including parallel vacuum), but that would reduce how much code
duplication we can avoid for index builds.
Thoughts?
Thanks,
Lukas
--
Lukas Fittl
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0001-instrumentation-Use-Instrumentation-struct-for-pa.patch | application/octet-stream | 30.9 KB |
| v1-0003-Unify-parallel-worker-handling-for-query-text.patch | application/octet-stream | 13.4 KB |
| v1-0002-Unify-parallel-worker-handling-for-instrumentatio.patch | application/octet-stream | 11.7 KB |
| v1-0004-Unify-parallel-index-build-infrastructure.patch | application/octet-stream | 46.1 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Ben Mejia | 2026-05-31 19:41:13 | [RFC] Secondary hash to split stuck hash-join batches |
| Previous Message | Andrey Borodin | 2026-05-31 18:27:12 | Re: Multi-Entry Indexing for GiST & SP-GiST |