RE: Parallel Apply

From: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: 'Tomas Vondra' <tomas(at)vondra(dot)me>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>
Subject: RE: Parallel Apply
Date: 2025-12-01 10:46:06
Message-ID: OSCPR01MB149667D359F65822748A4BC5DF5DBA@OSCPR01MB14966.jpnprd01.prod.outlook.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dear Tomas,

Thanks for seeing the thread and sorry for late response.
I had a PostgreSQL conference in Japan.

> However, the patch seems fairly large (~80kB, although a fair bit of
> that is comments). Would it be possible to split it into smaller chunks?
> Is there some "minimal patch", which could be moved to 0001, and then
> followed by improvements in 0002, 0003, ...? I sometimes do some
> "infrastructure" first, and the actual patch in the last part (simply
> using the earlier parts).
>
> I'm not saying it has to be split (or how exactly), but I personally
> find smaller patches easier to review ...

Yes, smaller patches are always better than huge monolith. I splitted the patch
into four patches - three of them introduces a mechanism to track dependencies
and wait until other transactions finish, and fourth patch launches parallel
workers with them. Each patch can be built and pass tests individually.
Two of them might be still large (-800 lines) but I hope this is helpful for
reviewers.

> Some comments / questions after looking at the patch today:

We would answer them after more analysis.

Best regards,
Hayato Kuroda
FUJITSU LIMITED

Attachment Content-Type Size
v3-0001-Introduce-new-type-of-logical-replication-message.patch application/octet-stream 6.0 KB
v3-0002-Introduce-a-shared-hash-table-to-store-paralleliz.patch application/octet-stream 8.7 KB
v3-0003-Introduce-a-local-hash-table-to-store-replica-ide.patch application/octet-stream 27.7 KB
v3-0004-Parallel-apply-non-streaming-transactions.patch application/octet-stream 46.5 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Mihail Nikalayeu 2025-12-01 10:49:37 Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements
Previous Message Aleksander Alekseev 2025-12-01 10:42:44 Re: Migrate to autoconf 2.72?