BUG #19549: Physical replication slot xmin value stuck

The following bug has been logged on the website:

Bug reference: 19549
Logged by: nitin mangnale
Email address: nmangnale2(at)gmail(dot)com
PostgreSQL version: Unsupported/Unknown
Operating system: Linux (ubuntu 22.04 LTS)
Description:

Hello PostgreSQL Community,

I am looking for assistance in investigating what appears to be an unusual
replication/xmin behavior on PostgreSQL 13.20.

Environment

- PostgreSQL Version: 13.20 (Community PostgreSQL)
- Replication Type: Physical Streaming Replication
- Primary Server: 1
- Standby Servers: 2
- Replication Slots: Physical replication slots
- One standby is used primarily as a reporting/read-only node.
- "hot_standby_feedback" is enabled.

Problem Description

One of our standby servers periodically causes a physical replication slot
on the primary to retain a very old "xmin". The "age(xmin)" continuously
increases over time, preventing vacuum from advancing normally on the
primary.

The unusual part is that we cannot identify any backend or query responsible
for holding the snapshot.

The issue only affects one standby. The second standby remains healthy.

Restarting the affected standby immediately clears the problem, but after
several days the same behavior returns.

Observed Behavior

On the primary:

SELECT now(),
slot_name,
slot_type,
active,
xmin,
age(xmin),
catalog_xmin,
restart_lsn
FROM pg_replication_slots
ORDER BY age(xmin) DESC NULLS LAST;

Example output:

slot_name : oseu2012766
slot_type : physical
active : true
xmin : 788928341
age(xmin) : 5419501 (continuously increasing)
restart_lsn : 1835DD/EB000000

The second standby slot behaves normally:

slot_name : oseu2012755
age(xmin) : ~6000

---

On the primary:

SELECT application_name,
backend_xmin,
age(backend_xmin),
state,
sync_state
FROM pg_stat_replication;

Output:

application_name | backend_xmin | state
-----------------+--------------+----------
walreceiver | NULL | streaming
walreceiver | NULL | streaming

"backend_xmin" is NULL for both standby connections.

---

On the affected standby:

We checked:

- pg_stat_activity
- long-running transactions
- idle in transaction sessions
- backend_xmin
- xact_start

No session appears to hold the reported xmin.

Example:

SELECT pid,
usename,
application_name,
backend_xmin,
age(backend_xmin),
state,
query
FROM pg_stat_activity
WHERE backend_xmin = 788928341;

Returns:

0 rows

We also reviewed all active sessions and found only short-running reporting
queries (generally under 20 seconds).

---

Replication Status

- Replication lag: None
- WAL receiver: Healthy
- WAL replay: Normal
- Physical replication slot remains active

---

Database Conflicts

SELECT *
FROM pg_stat_database_conflicts;

Output:

confl_snapshot = 0
confl_lock = 0
confl_deadlock = 0
confl_bufferpin = 20

---

Actions Already Performed

We have verified:

- No long-running queries
- No idle-in-transaction sessions
- No prepared transactions
- No replication lag
- No obvious replication slot issues
- No backend_xmin matching the slot xmin
- WAL receiver appears healthy
- Reporting queries are relatively short-lived

Restarting only the affected standby immediately releases the pinned xmin
and vacuum resumes normally.

However, after a few days, the same standby again begins holding an old
xmin.

The second standby does not exhibit this behavior.

Questions

1. Is this a known issue or edge case in PostgreSQL 13.20 involving physical
replication slots or hot_standby_feedback?

2. Is it expected for "pg_replication_slots.xmin" to remain pinned while
"pg_stat_replication.backend_xmin" is NULL?

3. Are there any known bugs in PostgreSQL 13.x related to stale standby
feedback or snapshot lifecycle that match this behavior?

4. Are there any additional internal views, debugging techniques, or GUC
parameters that would help identify the source of the pinned xmin?

5. Would upgrading to a newer PostgreSQL major version be expected to
resolve this behavior if it is related to snapshot or hot standby feedback
handling?

At this point we are considering rebuilding the affected standby, but we
would like to understand the root cause before doing so.

Any guidance would be greatly appreciated.

Thank you.