RE: Introduce XID age based replication slot invalidation

From: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>
To: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Bertrand Drouvot <bertranddrouvot(dot)pg(at)gmail(dot)com>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Srinath Reddy Sadipiralla <srinath2133(at)gmail(dot)com>, SATYANARAYANA NARLAPURAM <satyanarlapuram(at)gmail(dot)com>, "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, John H <johnhyvr(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: Introduce XID age based replication slot invalidation
Date: 2026-09-01 04:13:10
Message-ID: TY4PR01MB177185E08A15401D1A1910B6994A82@TY4PR01MB17718.jpnprd01.prod.outlook.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tuesday, September 1, 2026 11:20 AM Zhijie Hou (Fujitsu) <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
> On Friday, August 28, 2026 8:17 AM Bharath Rupireddy
> <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
> > Since the pg_conflict_detection slot is internally created and
> > maintained for the user, I don't think invalidating it (by terminating
> > the launcher and asking every apply worker to stop conflict retention)
> > is the right approach. After invalidating, the slot cannot be dropped,
> > and the conflict detection feature will no longer be working
> > correctly, and the launcher will not error out for invalidated slots.
> > The user doesn't even have an easy way to deal with this situation.
> > They would have already set retain_dead_tuples and
> > max_retention_duration for the subscription, but they now have no way
> > to get conflict detection to work again.
> >
> > That said, one of the design goals for the XID-age based slot
> > invalidation feature is that no slot blocks vacuum indefinitely.
> >
> > My proposal is to check the age of each worker's
> > oldest_nonremovable_xid against the XID-age GUC in each apply worker
> > in should_stop_conflict_info_retention(). If it is aged, stop the
> > retention. This mechanism is simple to reason about and fits well with
> > the backoff we already have with max_retention_duration. This keeps
> > the slot valid but backs off from holding the vacuum.
>
> I am wondering whether we could choose a simpler way. That is, change the
> launcher to not acquire the slot when it's invalidated. This way the user has
> the option to drop the slot and disable retain_dead_tuples, or just drop the
> slot and the launcher will later re-create it if retain_dead_tuples is still
> enabled.

Note that this might require changes to allow users to drop reserved slots (the
pg_conflict_detection slot is reserved by design, so users cannot acquire it).
So this might not be practical.

If we want to avoid this change, a simpler approach is to let the launcher
auto-drop the invalidated slot, recreate it, and log a WARNING.

>
> This also sidesteps the problem Nisha pointed out with the worker-side check:
> when a subscription is disabled or its worker can't start, there is no worker to
> run the backoff while the launcher keeps the slot's xmin frozen. With this
> approach the slot is simply invalidated like any other.
>
> About the risk of missing conflict detections due to this invalidation: I think
> this is no different from a logical slot on the publisher being invalidated due to
> similar reasons (XID age or WAL size), where replication stops and changes
> are missed - the user has to manually drop and re-create the slot to resume
> replication. (The conflict detections are anyway missed even if we stop the
> retention in this case)
>
> We could document that the slot can be invalidated if the XID-age GUC is
> configured, and that the user can drop the slot manually to recover.

Best Regards,
Zhijie Hou

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Nisha Moond 2026-09-01 04:27:07 Re: Proposal: Conflict log history table for Logical Replication
Previous Message Bharath Rupireddy 2026-09-01 04:10:37 Re: [PATCH] Fix autovacuum freeze bug where weight can lower a table's freeze score