Re: Parallel Apply

On Mon, Dec 1, 2025 at 4:16 PM Hayato Kuroda (Fujitsu)
<kuroda(dot)hayato(at)fujitsu(dot)com> wrote:
>
> Dear Tomas,
>
> Thanks for seeing the thread and sorry for late response.
> I had a PostgreSQL conference in Japan.
>
> > However, the patch seems fairly large (~80kB, although a fair bit of
> > that is comments). Would it be possible to split it into smaller chunks?
> > Is there some "minimal patch", which could be moved to 0001, and then
> > followed by improvements in 0002, 0003, ...? I sometimes do some
> > "infrastructure" first, and the actual patch in the last part (simply
> > using the earlier parts).
> >
> > I'm not saying it has to be split (or how exactly), but I personally
> > find smaller patches easier to review ...
>
> Yes, smaller patches are always better than huge monolith. I splitted the patch
> into four patches - three of them introduces a mechanism to track dependencies
> and wait until other transactions finish, and fourth patch launches parallel
> workers with them. Each patch can be built and pass tests individually.
> Two of them might be still large (-800 lines) but I hope this is helpful for
> reviewers.
>
> > Some comments / questions after looking at the patch today:
>
> We would answer them after more analysis.

I was just going through the commit messages of all the patches, I
could not understand the last line of below paragraph in v3-0004, what
do you mean by the last line which says "after which the leader
updates the
hash entry with the current xid"?

"The leader maintains a local hash table, using the remote change's replica
identity column values and relid as keys, with remote transaction IDs as values.
Before sending changes to the parallel apply worker, the leader computes a hash
using RI key values and the relid of the current change to search the hash
table. If an existing entry is found, the leader tells the parallel worker
to wait for the remote xid in the hash entry, after which the leader updates the
hash entry with the current xid."

--
Regards,
Dilip Kumar
Google