RE: Parallel Apply

From: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: 'Dilip Kumar' <dilipbalaut(at)gmail(dot)com>
Cc: Tomas Vondra <tomas(at)vondra(dot)me>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Subject: RE: Parallel Apply
Date: 2025-12-02 07:19:13
Message-ID: OSCPR01MB149662F6753E26278F340438CF5D8A@OSCPR01MB14966.jpnprd01.prod.outlook.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dear Dilip,

> I was just going through the commit messages of all the patches, I
> could not understand the last line of below paragraph in v3-0004, what
> do you mean by the last line which says "after which the leader
> updates the
> hash entry with the current xid"?
>
> "The leader maintains a local hash table, using the remote change's replica
> identity column values and relid as keys, with remote transaction IDs as values.
> Before sending changes to the parallel apply worker, the leader computes a hash
> using RI key values and the relid of the current change to search the hash
> table. If an existing entry is found, the leader tells the parallel worker
> to wait for the remote xid in the hash entry, after which the leader updates the
> hash entry with the current xid."

This meant if two transactions had changes for the same RI, lastly committed
transaction's XID could be stored here. In other words, each local hash entry always
has the latest XID which modifies a key (RI).

Assuming that there are three transactions T1->T2->T3 and they modify the same
tuple. When subscriber applies T3, it should wait till T2 is committed, not T1.
XID of the entry should be updated for implementing it.

I tried to rephrase that line a bit, how do you feel? All patches are attached
to keep CI happy.

Best regards,
Hayato Kuroda
FUJITSU LIMITED

Attachment Content-Type Size
v4-0001-Introduce-new-type-of-logical-replication-message.patch application/octet-stream 6.0 KB
v4-0002-Introduce-a-shared-hash-table-to-store-paralleliz.patch application/octet-stream 8.7 KB
v4-0003-Introduce-a-local-hash-table-to-store-replica-ide.patch application/octet-stream 27.7 KB
v4-0004-Parallel-apply-non-streaming-transactions.patch application/octet-stream 46.5 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2025-12-02 07:27:50 Re: more C99 cleanup
Previous Message Antonin Houska 2025-12-02 07:18:48 Index functions and INDEX_CREATE_SKIP_BUILD