RE: Parallel Apply

From: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>
To: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
Cc: shveta malik <shveta(dot)malik(at)gmail(dot)com>, Tomas Vondra <tomas(at)vondra(dot)me>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Andrei Lepikhov <lepihov(at)gmail(dot)com>, wenhui qiu <qiuwenhuifx(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: RE: Parallel Apply
Date: 2026-08-21 03:21:41
Message-ID: TY4PR01MB1771824F83BBB3634B837A75894A32@TY4PR01MB17718.jpnprd01.prod.outlook.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wednesday, July 15, 2026 1:03 PM Kuroda, Hayato <kuroda(dot)hayato(at)fujitsu(dot)com> wrote:
> I found cfbot cannot accept the v21 patch set, please see the updated
> version.

Here is the updated patch set.

In this version, I've re-split the patches to make each one implement a more
specific, self-contained functionality that is a necessary part of supporting
parallel apply. I also rewrote the commit messages for each patch and moved the
overall design summary to the 0001 patch, so reviewers can understand the
architecture by reading just the first commit message.

In addition, I did quite a bit of refactoring and simplification to make the
code easier to read, and added more comments where needed.

For dependency tracking, I replaced the old change-count-based cleanup with a
change-size-based memory limit control. If the tuples saved for dependency
tracking exceed the limit, parallel apply is suspended and subsequent
transactions are applied by the leader until the preceding transaction commits
and the cached dependency changes are cleaned up.

In 0010, fixed a bug where the DML operation was not marked as parallel unsafe.

For local (subscriber-side) unique key and foreign key dependency tracking,
Kuroda-san and I worked on supporting partitioned tables. For partitioned
tables, the leader now collects unique indexes from the leaf partitions rather
than the parent table. Patches 0011 and 0012 implement this support.

Best Regards,
Zhijie Hou

Attachment Content-Type Size
v23-0001-Support-remote-relation-synchronization-for-para.patch application/octet-stream 20.9 KB
v23-0012-Support-dependency-tracking-via-local-foreign-ke.patch application/octet-stream 56.7 KB
v23-0011-Support-dependency-tracking-via-local-unique-ind.patch application/octet-stream 49.2 KB
v23-0010-Handle-unsafe-parallel-apply-cases-due-to-non-im.patch application/octet-stream 21.7 KB
v23-0009-Track-dependencies-for-streamed-transactions.patch application/octet-stream 9.8 KB
v23-0008-support-2PC.patch application/octet-stream 13.5 KB
v23-0007-Bound-dependency-tracking-memory-with-serial-app.patch application/octet-stream 10.1 KB
v23-0006-Parallel-apply-non-streaming-transactions.patch application/octet-stream 44.2 KB
v23-0005-Support-dependency-tracking-via-remote-replica-i.patch application/octet-stream 27.4 KB
v23-0004-Prepare-commit-order-preservation-for-parallel-a.patch application/octet-stream 8.6 KB
v23-0003-Introduce-a-shared-hash-table-to-store-paralleli.patch application/octet-stream 9.3 KB
v23-0002-Prepare-flush-position-tracking-for-parallel-app.patch application/octet-stream 14.3 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Sami Imseih 2026-08-21 03:34:22 postgres_fdw: Fix flaky push down FUNCTION RTE test
Previous Message Gabriele Bartolini 2026-08-21 03:07:42 Tracking role modification timestamps in pg_authid / pg_roles