| From: | "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com> |
|---|---|
| To: | "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Bertrand Drouvot <bertranddrouvot(dot)pg(at)gmail(dot)com> |
| Cc: | Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Srinath Reddy Sadipiralla <srinath2133(at)gmail(dot)com>, SATYANARAYANA NARLAPURAM <satyanarlapuram(at)gmail(dot)com>, "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, John H <johnhyvr(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
| Subject: | RE: Introduce XID age based replication slot invalidation |
| Date: | 2026-09-01 04:13:10 |
| Message-ID: | TY4PR01MB177185E08A15401D1A1910B6994A82@TY4PR01MB17718.jpnprd01.prod.outlook.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Tuesday, September 1, 2026 11:20 AM Zhijie Hou (Fujitsu) <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
> On Friday, August 28, 2026 8:17 AM Bharath Rupireddy
> <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
> > Since the pg_conflict_detection slot is internally created and
> > maintained for the user, I don't think invalidating it (by terminating
> > the launcher and asking every apply worker to stop conflict retention)
> > is the right approach. After invalidating, the slot cannot be dropped,
> > and the conflict detection feature will no longer be working
> > correctly, and the launcher will not error out for invalidated slots.
> > The user doesn't even have an easy way to deal with this situation.
> > They would have already set retain_dead_tuples and
> > max_retention_duration for the subscription, but they now have no way
> > to get conflict detection to work again.
> >
> > That said, one of the design goals for the XID-age based slot
> > invalidation feature is that no slot blocks vacuum indefinitely.
> >
> > My proposal is to check the age of each worker's
> > oldest_nonremovable_xid against the XID-age GUC in each apply worker
> > in should_stop_conflict_info_retention(). If it is aged, stop the
> > retention. This mechanism is simple to reason about and fits well with
> > the backoff we already have with max_retention_duration. This keeps
> > the slot valid but backs off from holding the vacuum.
>
> I am wondering whether we could choose a simpler way. That is, change the
> launcher to not acquire the slot when it's invalidated. This way the user has
> the option to drop the slot and disable retain_dead_tuples, or just drop the
> slot and the launcher will later re-create it if retain_dead_tuples is still
> enabled.
Note that this might require changes to allow users to drop reserved slots (the
pg_conflict_detection slot is reserved by design, so users cannot acquire it).
So this might not be practical.
If we want to avoid this change, a simpler approach is to let the launcher
auto-drop the invalidated slot, recreate it, and log a WARNING.
>
> This also sidesteps the problem Nisha pointed out with the worker-side check:
> when a subscription is disabled or its worker can't start, there is no worker to
> run the backoff while the launcher keeps the slot's xmin frozen. With this
> approach the slot is simply invalidated like any other.
>
> About the risk of missing conflict detections due to this invalidation: I think
> this is no different from a logical slot on the publisher being invalidated due to
> similar reasons (XID age or WAL size), where replication stops and changes
> are missed - the user has to manually drop and re-create the slot to resume
> replication. (The conflict detections are anyway missed even if we stop the
> retention in this case)
>
> We could document that the slot can be invalidated if the XID-age GUC is
> configured, and that the user can drop the slot manually to recover.
Best Regards,
Zhijie Hou
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Nisha Moond | 2026-09-01 04:27:07 | Re: Proposal: Conflict log history table for Logical Replication |
| Previous Message | Bharath Rupireddy | 2026-09-01 04:10:37 | Re: [PATCH] Fix autovacuum freeze bug where weight can lower a table's freeze score |