| From: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> |
|---|---|
| To: | Ajin Cherian <itsajin(at)gmail(dot)com> |
| Cc: | Zhijie Hou <houzhijie22(at)gmail(dot)com>, Bertrand Drouvot <bertranddrouvot(dot)pg(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
| Subject: | Re: Add a hook for handling logical decoding messages on subscribers. |
| Date: | 2026-11-20 18:35:00 |
| Message-ID: | CAD21AoA91YO+72AMyaBYBjs1n9owoJG17K1k1nJwh-gqmO2w9w@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Wed, Nov 18, 2026 at 4:48 PM Ajin Cherian <itsajin(at)gmail(dot)com> wrote:
>
> On Tue, Nov 17, 2026 at 6:06 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> >
> > On Sun, Nov 15, 2026 at 9:13 AM Zhijie Hou <houzhijie22(at)gmail(dot)com> wrote:
> > >
> > I've attached the updated patches.
> >
>
> Hi Sawada-san,
>
> I found an issue in 0002 with non-transactional messages.
>
> I found this while reviewing the patch with Claude, which also helped
> write the TAP test below.
>
> In apply_handle_message(), the non-transactional branch commits the
> handler's work locally but calls replorigin_xact_clear(false) first,
> so the commit doesn't advance the replication origin. Until the
> publisher receives feedback that this commit has been flushed, both
> the origin and the slot's confirmed_flush point before the message.
> Feedback lags, since apply workers default to synchronous_commit =
> off. If the worker exits in that window, the message is decoded and
> sent again on restart, and the handler runs again. The easiest way to
> hit it is a non-transactional message followed by a transaction that
> conflicts on the subscriber. Every retry re-applies the message before
> failing on the conflict again. With wal_retrieve_retry_interval =
> 200ms I saw 42 handler invocations for a single message in 8 seconds.
> Handlers are documented as needing to be idempotent, but repeating
> once per retry seems avoidable, since normal transactions don't have
> this problem.
Good find. Since we already treat msg.lsn as the position the message
is applied at when reporting the flush position, it makes sense to
advance the origin to it as well, as we do for commits.
Given that we’ve agreed on the 0001 patch, I’m going to push it early
next week, barring any objections. I’d like to hold off on the 0002
patch to give it more review time.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
| Attachment | Content-Type | Size |
|---|---|---|
| v10-0001-Add-an-xid-argument-to-the-logical-decoding-mess.patch | text/x-patch | 20.3 KB |
| v10-0002-Add-a-hook-for-handling-logical-messages-on-subs.patch | text/x-patch | 123.8 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Gilles Darold | 2026-11-20 18:36:23 | Re: Backpatch for fix simplehash to preserve consistency in case of OOM |
| Previous Message | Masahiko Sawada | 2026-11-20 18:30:32 | Re: Use an index scan in RemoveSubscriptionRel() |