| From: | "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com> |
|---|---|
| To: | Peter Smith <smithpb2250(at)gmail(dot)com> |
| Cc: | shveta malik <shveta(dot)malik(at)gmail(dot)com>, "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Tomas Vondra <tomas(at)vondra(dot)me>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Andrei Lepikhov <lepihov(at)gmail(dot)com>, wenhui qiu <qiuwenhuifx(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> |
| Subject: | RE: Parallel Apply |
| Date: | 2026-06-15 11:20:12 |
| Message-ID: | TY4PR01MB1771875A7A64B139EF8C8D05C94E62@TY4PR01MB17718.jpnprd01.prod.outlook.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Tuesday, June 9, 2026 12:14 PM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
> Hi Hou-San.
>
> Some review comments for v19-0001 and v19-0002
>
Thanks for the comments.
> //////////
> v19-0001
>
> 1a.
> Maybe that ParallelizedTxnEntry should be moved to just immediately above
> 'dshash_parameters' because it seems to belong with that, and currently it is
> splitting the ParallelApplyWorkerEntry from the ParallelApplyTxnHash.
Moved as suggested.
>
> ~
>
> 1b.
> /parameters for/Parameters for/
>
> ~~~
Fixed.
> pa_attach_parallelized_txn_hash:
> 2a.
> This might be easier to read if rearranged to use if/else instead of having the
> early return.
I'm not in favor of this style, as deeply nested condition blocks are harder for
me to understand. Please see existing examples like init_dsm_registry,
initGlobalChannelTable, and logicalrep_launcher_attach_dshmem for the preferred
coding style.
>
> ~~~
>
> 2b.
> Can it be anything other than
> am_leader_apply_worker/am_parallel_apply_worker here? Should there be
> an Assert?
Only the leader and parallel apply workers need to coordinate in this context,
so I didn't add an Assert. Even if other processes use it, it will be a no-op,
which is fine.
>
> ~~~
>
> 2c.
> Since the `dsh_params` are already set up, shouldn't this code be using them?
>
> BEFORE
> dsa_create(LWTRANCHE_PARALLEL_APPLY_DSA);
>
> SUGGESTION
> dsa_create(dsh_params.tranche_id);
>
I think dsa_create and dsh_params are two different objects. The former is for
pure shared memory allocation, while the latter is part of a hash table.
Therefore, it's not necessary to pass dsh_params to dsa_create. We could even
use different LWTRANCHE for them if we wanted, but we don't have a special need
to create another one at the moment.
>
> //////////
> v19-0002
>
> ======
> .../replication/logical/applyparallelworker.
>
> 1.
> +void
> +pa_wait_for_depended_transaction(TransactionId xid) {
> +ParallelizedTxnEntry *txn_entry;
>
> The declaration of `txn_entry` can be done later within the loop where it is
> used.
Moved.
Attaching the new version patch set.
Apart from addressing above comments, I also did the following changes:
0003:
- Fix detection of dependencies on toasted columns by properly copying column
status.
- Ensure leader correctly handles table-wide dependencies when applying a
transaction directly.
- Prevent infinite loop by skipping self-dependencies when building the
dependency list.
- Expand test coverage for dependency tracking.
- Improve code readability through refactoring.
- Enhance comments for better documentation.
0008:
0009:
- Complete most TODOs and finalize the design to cover more cases. Add extensive
comments to explain the design.
- Add more tests to cover the new code paths.
Best Regards,
Hou zj
| Attachment | Content-Type | Size |
|---|---|---|
| v20-0001-Introduce-a-shared-hash-table-to-store-paralleli.patch | application/octet-stream | 9.1 KB |
| v20-0009-Support-dependency-tracking-via-local-foreign-ke.patch | application/octet-stream | 42.4 KB |
| v20-0008-Support-dependency-tracking-via-local-unique-ind.patch | application/octet-stream | 38.8 KB |
| v20-0007-Wait-applying-transaction-if-one-of-user-defined.patch | application/octet-stream | 11.9 KB |
| v20-0006-Track-dependencies-for-streamed-transactions.patch | application/octet-stream | 10.4 KB |
| v20-0005-support-2PC.patch | application/octet-stream | 13.8 KB |
| v20-0004-Parallel-apply-non-streaming-transactions.patch | application/octet-stream | 64.3 KB |
| v20-0003-Introduce-a-local-hash-table-to-store-replica-id.patch | application/octet-stream | 35.4 KB |
| v20-0002-Introduce-internal-messages-to-track-dependencie.patch | application/octet-stream | 12.7 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Henson Choi | 2026-06-15 11:29:19 | Re: LLVM JIT: any JIT-compiled query crashes (SIGILL) on a libLLVM 19 + ASAN build |
| Previous Message | Dilip Kumar | 2026-06-15 11:15:38 | Re: Proposal: Conflict log history table for Logical Replication |