| From: | 이민재 <mmj9808(at)gmail(dot)com> |
|---|---|
| To: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Subject: | [PROPOSAL] Natively expose primary key and NOT NULL flags in pgoutput Relation messages |
| Date: | 2026-07-06 12:04:51 |
| Message-ID: | CAL_7KJnnaMdCqD1mayhH7cv8GPwWmtrJ6e82CTxN9homy41Y6g@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi hackers,
Currently, the per-column flags in the pgoutput Relation message only
carry replica identity membership. Consumers that need the actual
primary key -- typically CDC tools computing deterministic record
keys -- are forced to query the catalogs over a separate connection.
This "side channel" is not tied to the WAL position being decoded.
This creates race conditions during schema changes, when the catalog
state runs ahead of the logical decoding stream. Debezium, for
example, documents this explicitly and carries best-effort code to
discard primary key columns that arrive "from the future" [1][2]. The
issue is especially pronounced when tables use REPLICA IDENTITY FULL,
which leaves downstream consumers completely blind to the actual
primary key via the WAL stream.
Because the decoding machinery already resolves the catalogs with a
historic snapshot, the server could provide this
point-in-time-correct information natively. I could not find a prior
discussion of this on the list.
Proposal: add an opt-in pgoutput parameter (e.g.,
extended_column_flags), similar in spirit to
publish_generated_columns, passed via the replication command
options. When enabled, it sets additional bits in the existing
per-column flags byte of the Relation message:
- one bit for primary key membership
- one bit for attnotnull
The flags byte currently uses only bit 0 (replica identity), so
existing consumers remain unaffected when the option is omitted.
Furthermore, with the not-null rework in PG 18 storing these
constraints in pg_constraint, this information is cleanly accessible
at the decoding level.
Does this sound like a reasonable direction? If the community is open
to this, I am happy to draft and submit a patch.
[1]
https://github.com/debezium/debezium/blob/main/debezium-connector-postgres/src/main/java/io/debezium/connector/postgresql/connection/pgoutput/PgOutputMessageDecoder.java
[2] https://github.com/debezium/dbz/issues/829
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Thomas Munro | 2026-07-06 12:09:00 | pg_threads.h take II |
| Previous Message | Heikki Linnakangas | 2026-07-06 11:58:12 | Re: remove unnecessary volatile qualifiers |