Re: Document prepared transaction lock waits with two_phase subscriptions

From: Bertrand Drouvot <bertranddrouvot(dot)pg(at)gmail(dot)com>
To: Zhijie Hou <houzhijie22(at)gmail(dot)com>
Cc: "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, "amit(dot)kapila16(at)gmail(dot)com" <amit(dot)kapila16(at)gmail(dot)com>, Nisha Moond <nisha(dot)moond412(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, shveta malik <shveta(dot)malik(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Subject: Re: Document prepared transaction lock waits with two_phase subscriptions
Date: 2026-11-16 09:36:12
Message-ID: avrOjINXdJs1XiwW@bdtpg
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

On Fri, Nov 13, 2026 at 11:58:10AM +0800, Zhijie Hou wrote:
>
> To recover from this state, the user can either cancel the waiting local
> session on the subscriber so the apply worker can proceed, or manually roll
> back the prepared transaction on the subscriber, disable two_phase until
> that transaction is replicated again after being committed on the publisher,
> and then re-enable two_phase if required. Considering the existing caution
> about prepared transactions and the available workarounds, we suggest
> documenting this locking hazard and how to resolve it for logical
> replication as well.

Yeah, +1 for documenting it.

> Please see the attached patch for the documentation changes.

=== 1

I wonder if it would also be worth mentioning a nonzero lock_timeout as a possible
mitigation too?

=== 2

+ logical replication apply worker can be completed only by that worker, see

s/can be completed only by that worker/is normally completed by the subscription's apply worker/?

=== 3

+ <command>ALTER TABLE</command>), it has to wait, and later lock
+ requests on that table

s/later lock requests/later conflicting lock requests/?

=== 4

+ The apply worker then cannot reach
+ the <command>COMMIT PREPARED</command> or <command>ROLLBACK
+ PREPARED</command> that would release the lock, and the sessions
+ wait for each other. This is not detected as a deadlock, so
+ replication stalls.

what about?

or <command>ROLLBACK PREPARED</command>. This situation is not detected as a
deadlock, so replication can remain stalled indefinitely.

> Apart from documenting this, Amit, Nisha, and I also discussed off-list
> whether the apply worker could detect and resolve this wait cycle
> automatically. One option we considered is to have the apply worker walk its
> lock blocking chain once a lock wait exceeds deadlock_timeout. Starting from
> the lock it is waiting for, the worker would trace conflicting lock holders
> and conflicting waiters ahead of it in the wait queue (similar to how
> pg_blocking_pids() finds blockers) to see whether the chain leads back to a
> prepared transaction of the same subscription, and if so, cancel the session
> waiting on that prepared transaction (or raise an error if the worker is
> directly blocked by its own prepared transaction). However, this effectively
> introduces a separate wait-graph traversal alongside the existing deadlock
> detector, which seems overly complex for this scenario.

Yeah, I agree. Teaching the lock manager about this logical dependency seems too
complex.

> We also thought about a much simpler check where, on timeout, the apply
> worker only inspects the lock it is currently waiting on and treats it as a
> wait cycle if a prepared transaction of the same subscription also holds a
> lock on that same table. Unfortunately, that does not cover multi-table
> cases.

Agreed. FWIW, lock_timeout on the local session handles this multi-table case too.

> Given these trade-offs, we prefer to document this locking hazard and the
> workarounds for now.

Another options I could see (because the doc changes explain the workarounds)
would be to help diagnose:

1/ improve log_lock_waits to identify a prepared transaction holding a lock by
its XID, instead of reporting it as process 0.

2/ improve the existing pg_locks example query so that it identifies locks
held by both newly prepared and recovered prepared transactions.

Thoughts?

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message shveta malik 2026-11-16 09:44:35 Initial table sync copies the whole table when it is no longer published
Previous Message Amit Kapila 2026-11-16 09:30:26 Re: Assertion failure in update_local_synced_slot(), slotsync.c:364