| From: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> |
|---|---|
| To: | Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> |
| Cc: | SATYANARAYANA NARLAPURAM <satyanarlapuram(at)gmail(dot)com>, "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, John H <johnhyvr(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
| Subject: | Re: Introduce XID age based replication slot invalidation |
| Date: | 2026-03-23 23:36:07 |
| Message-ID: | CAD21AoATM=Un8ejnbcDQ7q+CaCoxpkA7Ln9bacvQEoymqvjPug@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
On Mon, Mar 23, 2026 at 9:00 AM Bharath Rupireddy
<bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
>
> Hi,
>
> On Fri, Mar 20, 2026 at 11:29 PM SATYANARAYANA NARLAPURAM
> <satyanarlapuram(at)gmail(dot)com> wrote:
> >
> > Do you think we need different GUCs for catalog_xmin and xmin? If table bloat is a concern (not catalog bloat), then logical slots are not required to invalidate unless the cluster is close to wraparound.
>
> IMO the main purpose of max_slot_xid_age is to prevent XID wraparound.
> For bloat, I still think max_slot_wal_keep_size is the better choice.
>
> Where max_slot_xid_age is really useful is when the vacuum can't
> freeze because a replication slot (physical or logical) is holding
> back the XID horizon and the system is getting close to wraparound.
> Invalidating such a slot clears the way for vacuum. Setting
> max_slot_xid_age above vacuum_failsafe_age allows vacuum to waste
> cycles scanning tables it cannot freeze. Keeping max_slot_xid_age <=
> vacuum_failsafe_age (default 1.6B) prevents this by invalidating the
> slot before vacuum effort is wasted.
>
> As far as XID wraparound is concerned, both xmin and catalog_xmin need
> to be treated similarly. Either one can hold back freezing and push
> the system toward wraparound. So I don't think we need separate GUCs
> for xmin and catalog_xmin unless I'm missing something. One GUC
> covering both keeps things simple.
I've studied the discussion on this thread and the patch. I understand
the purpose of this feature and agree that it's useful especially in
cases where orphaned (physical or logical) replication slots prevent
the xmin from advancing and inactive_since based slot invalidation
might not fit.
And +1 for treating both the slot's xmin and catalog_xmin similarly
with the single GUC.
> >> I made the following design choice: try invalidating only once per
> >> vacuum cycle, not per table. While this keeps the cost of checking
> >> (incl. the XidGenLock contention) for invalidation to a minimum when
> >> there are a large number of tables and replication slots, it can be
> >> less effective when individual tables/indexes are large. Invalidating
> >> during checkpoints can help to some extent with the large table/index
> >> cases. But I'm open to thoughts on this.
> >
> > It may not solve the intent when the vacuum cycle is longer, which one can expect on a large database particularly when there is heavy bloat.
>
> This design choice boils down to the following: a database instance
> having either 1/ a large number of small tables or 2/ large tables.
> From my experience, I have seen both cases but mostly case 2 (others
> can correct me). In this context, having an XID age based slot
> invalidation check once per relation makes sense. However, I'm open to
> more thoughts here.
ISTM that checking the XID-based slot invalidation per table would be
more bullet-proof and cover many cases. How about checking the
XID-based slot invalidation opportunity only when the OldestXmin is
older than the new GUC? For example, we can do this check in
heap_vacuum_rel() based on the VacuumCutoffs returned by
vacuum_get_cutoffs(). If we invalidate at least one slot for its XID,
we can re-compute the OldestXmin.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
| From | Date | Subject | |
|---|---|---|---|
| Next Message | David Rowley | 2026-03-24 00:12:17 | Re: Fix "could not find memoization table entry" |
| Previous Message | KAZAR Ayoub | 2026-03-23 23:36:03 | Re: Add pg_stat_vfdcache view for VFD cache statistics |