Re: BUG #19616: pgoutput sends stream abort ('A') to clients that did not enable streaming

There are two shapes for the fix and I would like opinions on which is
preferred. Both are attached, they are mutually exclusive, and each
carries the same regress case.

0001-Don-t-mark-discarded-aborted-subxacts-as-streamed.patch adds the
guard at the call site in ReorderBufferTruncateTXN. It is a one-line
change and it does not touch the shared helper.

v2-0001-Don-t-mark-discarded-aborted-subxacts-as-streamed.patch moves the
rule into ReorderBufferMaybeMarkTXNStreamed itself. A subxact is marked
only when it has changes and its top-level xact is already streamed. This
states the invariant in one place and reads well on its own.

Thoughts?

чт, 13 авг. 2026 г. в 22:56, Andrey Rachitskiy <pl0h0yp1(at)gmail(dot)com>:

> Hi, Tyler!
>
> Thanks for the report and analysis.
>
> ReorderBufferTruncateTXN is used both after streaming and when discarding
> already-aborted transactions at eviction. Streaming callers mark the
> top-level xact before truncating. The abort-discard path does not mark
> it. The attached patch marks a subxact as streamed only when the
> top-level xact already is. A later abort then does not emit stream_abort
> for XIDs that were never sent downstream.
>
> Patch with regress tests attached.
>
> чт, 13 авг. 2026 г. в 20:30, PG Bug reporting form <noreply(at)postgresql(dot)org
> >:
>
>> The following bug has been logged on the website:
>>
>> Bug reference: 19616
>> Logged by: Tyler Smart
>> Email address: tyler(at)smarts(dot)io
>> PostgreSQL version: 18.4
>> Operating system: Linux (Docker; also Google Cloud SQL)
>> Description:
>>
>> Since PostgreSQL 18, pgoutput can send a Stream Abort ('A') message to a
>> client that connected with proto_version 1 and never enabled streaming.
>> Protocol 1 clients do not implement the stream message set, so consumers
>> fail on it: Debezium (and tools that embed it, like Airbyte) dies with
>> "Unsupported message type: A", and because the crash repeats at the same
>> WAL
>> position on every restart, the slot stops advancing until
>> max_slot_wal_keep_size invalidates it.
>>
>> I reproduced this on 18.4 (Debian 18.4-1.pgdg13+1, official Docker image)
>> and on Cloud SQL 18.1. The identical script does not reproduce it on
>> 17.10.
>>
>> The trigger needs three conditions in one decode run:
>>
>> 1. logical_decoding_work_mem is exceeded, so eviction runs.
>> 2. The eviction candidate has already aborted in clog.
>> 3. That transaction has a subtransaction with changes still in memory.
>>
>> From reading REL_18_STABLE, the cause appears to be commit 072ee847ad4
>> ("Skip logical decoding of already-aborted transactions"). It added
>> ReorderBufferCheckAndTruncateAbortedTXN, which discards aborted
>> transactions
>> at eviction time via ReorderBufferTruncateTXN. That function marks every
>> subtransaction that still has in-memory changes as streamed
>> (ReorderBufferMaybeMarkTXNStreamed, in the subtxn loop near
>> reorderbuffer.c:1675) without checking whether the connection streams at
>> all. The top-level transaction is handled correctly, since its marking
>> happens at call sites guarded by the streaming flag. Only the
>> subtransaction
>> marking is unconditional.
>>
>> When decoding later reaches the abort record, ReorderBufferAbort sees
>> rbtxn_is_streamed on the subtransaction and invokes the stream_abort
>> callback (near reorderbuffer.c:3092). pgoutput_stream_abort is guarded
>> only
>> by assertions, so production builds write 'A' onto a proto_version 1
>> stream. I expect a cassert build to fail Assert(rbtxn_is_streamed(toptxn))
>> there instead, since the top-level transaction is not marked, though I
>> have
>> not verified that.
>>
>> client that connected with proto_version 1 and never enabled streaming.
>> Protocol 1 clients do not implement the stream message set, so consumers
>> fail on it: Debezium (and tools that embed it, like Airbyte) dies with
>> "Unsupported message type: A", and because the crash repeats at the same
>> WAL
>> position on every restart, the slot stops advancing until
>> max_slot_wal_keep_size invalidates it.
>>
>> I reproduced this on 18.4 (Debian 18.4-1.pgdg13+1, official Docker image)
>> and on Cloud SQL 18.1. The identical script does not reproduce it on
>> 17.10.
>>
>> The trigger needs three conditions in one decode run:
>>
>> 1. logical_decoding_work_mem is exceeded, so eviction runs.
>> 2. The eviction candidate has already aborted in clog.
>> 3. That transaction has a subtransaction with changes still in memory.
>>
>> From reading REL_18_STABLE, the cause appears to be commit 072ee847ad4
>> ("Skip logical decoding of already-aborted transactions"). It added
>> ReorderBufferCheckAndTruncateAbortedTXN, which discards aborted
>> transactions
>> at eviction time via ReorderBufferTruncateTXN. That function marks every
>> subtransaction that still has in-memory changes as streamed
>> (ReorderBufferMaybeMarkTXNStreamed, in the subtxn loop near
>> reorderbuffer.c:1675) without checking whether the connection streams at
>> all. The top-level transaction is handled correctly, since its marking
>> happens at call sites guarded by the streaming flag. Only the
>> subtransaction
>> marking is unconditional.
>>
>> When decoding later reaches the abort record, ReorderBufferAbort sees
>> rbtxn_is_streamed on the subtransaction and invokes the stream_abort
>> callback (near reorderbuffer.c:3092). pgoutput_stream_abort is guarded
>> only
>> by assertions, so production builds write 'A' onto a proto_version 1
>> stream. I expect a cassert build to fail Assert(rbtxn_is_streamed(toptxn))
>> there instead, since the top-level transaction is not marked, though I
>> have
>> not verified that.
>>
>> Self-contained reproduction (the SQL decoding interface acts as a
>> non-streaming client, so no replication client is needed):
>>
>> docker run -d -e POSTGRES_PASSWORD=pw postgres:18 -c wal_level=logical
>>
>> CREATE TABLE t(id int, filler text);
>> CREATE PUBLICATION pub FOR TABLE t;
>> SELECT pg_create_logical_replication_slot('s', 'pgoutput');
>>
>> BEGIN;
>> SAVEPOINT sp;
>> INSERT INTO t VALUES (0, 'subtransaction-change');
>> RELEASE SAVEPOINT sp;
>> INSERT INTO t SELECT g, repeat('x', 1000) FROM generate_series(1, 5000)
>> g;
>> ROLLBACK;
>>
>> INSERT INTO t VALUES (1, 'after');
>>
>> SET logical_decoding_work_mem = '64kB';
>> SELECT chr(get_byte(data,0)) AS msgtype, count(*)
>> FROM pg_logical_slot_peek_binary_changes('s', NULL, NULL,
>> 'proto_version','1','publication_names','pub')
>> GROUP BY 1 ORDER BY 2 DESC;
>>
>> Actual output on 18.4:
>>
>> msgtype | count
>> ---------+-------
>> B | 1
>> R | 1
>> C | 1
>> I | 1
>> A | 1
>>
>> Expected: no A row. A proto_version 1 client must never receive stream
>> messages, and an aborted transaction should produce no output at all.
>> PG 17.10 produces the expected output with the same script, as does 18.4
>> when logical_decoding_work_mem is raised enough that eviction never fires.
>>
>> Impact: any protocol 1 consumer on a busy PG 18 server can hit this with a
>> single canceled or deadlocked transaction that used savepoints, decoded
>> while the buffer is past logical_decoding_work_mem. We hit it in
>> production
>> through Debezium, where the retry loop pinned the slot until Postgres
>> invalidated it. Raising logical_decoding_work_mem only lowers the
>> probability.
>>
>> Suggested direction: ReorderBufferTruncateTXN should mark subtransactions
>> as streamed only when truncating on behalf of streaming, the same way the
>> top-level marking is already gated, or the abort-discard path should skip
>> the marking entirely.
>>
>>
>>
>>
>>

--
Regards,
Rachitskiy Andrey