RE: Parallel Apply

From: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>
To: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>, "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>
Cc: Tomas Vondra <tomas(at)vondra(dot)me>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Andrei Lepikhov <lepihov(at)gmail(dot)com>, wenhui qiu <qiuwenhuifx(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: RE: Parallel Apply
Date: 2026-04-16 16:59:25
Message-ID: TYRPR01MB14195DCAC65FBEBBB896FB3B194232@TYRPR01MB14195.jpnprd01.prod.outlook.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Friday, April 17, 2026 12:05 AM Zhijie Hou (Fujitsu) <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
>
> On Tuesday, April 14, 2026 9:00 PM Kuroda, Hayato/黒田 隼人
> <kuroda(dot)hayato(at)fujitsu(dot)com> wrote:
> >
> > Other comments were addressed accordingly, please see attached patch set.
>
> I started reviewing patches 0001-0004 myself, aiming to add comments where
> the design is not straightforward and to identify and fix any clearly incorrect
> behavior.
>
> Here is the updated patch set with the following improvements:
>
> * Cosmetic changes in 0001-0004
> * Additional comments in 0001-0004
> * Code simplification by merging unnecessary static functions
> * Removal of function exports left over from the POC version that are no
> longer needed
> * Got rid of XLogRecPtrIsInvalid()
> * Fixed buggy behavior in partial serialization mode, including:
> 1) The leader did not serialize the dependency on the last committed
> transaction
> 2) The parallel apply worker could not identify internal messages in
> spooled changes
> 3) An assertion failure in maybe_start_skipping_changes()
> * Added one test for serialization and restore non-streaming transactions in
> 0004.
>
> Thanks to Kuroda-San for discussing these changes internally with me.

I noticed a CFbot failure caused by a missing identification of the internal
message code, sorry for that. Here are the updated patches to fix it.

Best Regards,
Hou zj

Attachment Content-Type Size
v14-0009-Support-dependency-tracking-via-foreign-keys.patch application/octet-stream 16.7 KB
v14-0001-Introduce-internal-messages-to-track-dependencie.patch application/octet-stream 9.5 KB
v14-0002-Introduce-a-shared-hash-table-to-store-paralleli.patch application/octet-stream 9.3 KB
v14-0003-Introduce-a-local-hash-table-to-store-replica-id.patch application/octet-stream 30.5 KB
v14-0004-Parallel-apply-non-streaming-transactions.patch application/octet-stream 55.8 KB
v14-0005-support-2PC.patch application/octet-stream 14.2 KB
v14-0006-Track-dependencies-for-streamed-transactions.patch application/octet-stream 11.0 KB
v14-0007-Wait-applying-transaction-if-one-of-user-defined.patch application/octet-stream 11.8 KB
v14-0008-Support-dependency-tracking-via-local-unique-ind.patch application/octet-stream 23.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Fujii Masao 2026-04-16 17:09:32 Re: Fix tab completion after EXCEPT (...) in IMPORT FOREIGN SCHEMA
Previous Message Lakshmi N 2026-04-16 16:46:45 Re: Reject invalid databases in pg_get_database_ddl()