REPACK (CONCURRENTLY) loses missing values of columns added without a rewrite

From: Sami Imseih <samimseih(dot)pg(at)gmail(dot)com>
To: Postgres hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: REPACK (CONCURRENTLY) loses missing values of columns added without a rewrite
Date: 2026-09-22 18:01:59
Message-ID: CAN12+Y+NJwr5EqKVrHpPD=5+b_NRAOzU9fhrVDO3H46x2WRxoA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

REPACK (CONCURRENTLY) is not correctly accounting for missing values when
applying decoded changes to the transient relation (NewHeap). This is
because it is using the NewHeap descriptor, which intentionally has no
missing values when it is formed in make_new_heap().

This can leave a column with NULL instead of its default value, even if the
column is NOT NULL. It only happens today when an UPDATE during the repack
fires a BEFORE UPDATE trigger that returns OLD, so the new row version
reuses a tuple written before the ALTER TABLE ... ADD COLUMN ... DEFAULT.

```
# session 1
CREATE EXTENSION injection_points;
CREATE TABLE t(i int primary key, j int);
INSERT INTO t VALUES (1,1),(2,2),(3,3);
ALTER TABLE t ADD COLUMN c int NOT NULL DEFAULT 42;
CREATE FUNCTION retold() RETURNS trigger LANGUAGE plpgsql AS 'BEGIN
RETURN OLD; END';
CREATE TRIGGER tr BEFORE UPDATE ON t FOR EACH ROW EXECUTE FUNCTION retold();
SELECT injection_points_attach('repack-concurrently-before-lock', 'wait');
REPACK (CONCURRENTLY) t;

# session 2
UPDATE t SET j = j WHERE i IN (1,2);
SELECT injection_points_wakeup('repack-concurrently-before-lock');
SELECT injection_points_detach('repack-concurrently-before-lock');
SELECT * FROM t;
```

The final SELECT * FROM t; results in

```
i | j | c
---+---+----
3 | 3 | 42
1 | 1 |
2 | 2 |
(3 rows)
```

where i = 1 and 2 are missing the value of the default column.

The same issue has been found previously in different cases. 16828d5c027
made every consumer of a stored tuple responsible for the missing values,
ba9f18abd36 worked around one such case by expanding the tuple in
MaterializeTupleForTrigger(), and 20d3fe9009d replaced that workaround by
using the real tuple descriptor instead.

The attached patch takes the same approach as 20d3fe9009d, which is to use
the real tuple descriptor rather than expand the tuple. It deforms the decoded
tuples with the descriptor of the relation they were decoded from, which is
the only one that has the missing values. Tuples formed with it are still
valid for the transient relation, whose attributes are a copy of the source
relation ones. An isolation test is included.

My first thought was to expand the decoded tuple with heap_expand_tuple(),
but that is the workaround of ba9f18abd which 20d3fe9009d got rid of.
heap_expand_tuple() has had no caller in the tree since then, and
minimal_expand_tuple() has had none since 4da597edf1b, so as a separate
cleanup for v20 it may be worth removing both.

I will add this to the v19 open items, as I think it should be backpatched.

--
Sami Imseih
Amazon Web Services

Attachment Content-Type Size
v1-0001-Fix-REPACK-CONCURRENTLY-for-columns-added-without.patch application/octet-stream 9.5 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Ayush Tiwari 2026-09-22 18:09:50 Re: [PATCH] Two remaining shmem attachment issues in single-user mode
Previous Message vignesh C 2026-09-22 17:58:40 Re: sequencesync worker race with REFRESH SEQUENCES