| From: | Michael Malis <malis(at)pgrust(dot)com> |
|---|---|
| To: | pgsql-hackers(at)postgresql(dot)org |
| Subject: | Preferred way to submit a very large batch of bugs? |
| Date: | 2026-11-03 02:06:23 |
| Message-ID: | CANVD+MDAvscHYpg9JKZeDVVL8pnqfjLttahGeJvZmmEvRzhFvA@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi all,
I know the mailing lists have been hit with a flood of automated bug
reports lately and
the last thing I want to do is create more headaches for the team.
Over the past few
months, I've been building a suite of tools for finding bugs. That
work has surfaced
over 1000 previously unreported bugs ranging from crashes, to
corruption, to MVCC
violations. I've included three of the bugs so you can get a sense of
some of the findings.
What's the least disruptive way for me to submit these issues? I've prepared a
searchable catalog that makes it easy to browse through the bugs based on the
category and severity of the issue, but I wanted to reach out first
before sharing anything.
Cheers,
Michael
** Hot standby query gets wrong results from index built by plain
CREATE INDEX **
With hot_standby_feedback = off (the default), a REPEATABLE READ
transaction on a hot standby can silently observe two different row
counts for the same query within a single snapshot after a plain CREATE
INDEX runs on the primary. Specifically, if rows visible to the standby
snapshot are deleted and committed on the primary and a new index is
then built on the table (without VACUUM or heap pruning, so all heap
tuples remain on the page), the primary treats the deleted tuples as
HEAPTUPLE_DEAD and omits them from the new index. Once the index build
is replayed on the standby, a sequential scan in the open REPEATABLE
READ transaction still returns all rows visible to its snapshot, whereas
an index scan or index-only scan using the new index omits the deleted
rows. No recovery conflict is raised even with
max_standby_streaming_delay at its default of 30s, breaking the
REPEATABLE READ guarantee that a transaction sees a consistent snapshot
unless canceled by a recovery conflict.
Reproduction (primary with a streaming hot standby, default
hot_standby_feedback = off and max_standby_streaming_delay = 30s or -1):
-- Primary:
CREATE TABLE t (id int, k int) WITH (autovacuum_enabled = off, fillfactor = 50);
INSERT INTO t SELECT g, g FROM generate_series(1, 1000) g;
-- Standby (after replay):
BEGIN ISOLATION LEVEL REPEATABLE READ;
SET enable_indexscan = off;
SET enable_bitmapscan = off;
SET enable_indexonlyscan = off;
SELECT count(*) FROM t WHERE k > 0;
-- actual: 1000
-- Primary:
DELETE FROM t WHERE id <= 500;
CREATE INDEX t_k ON t(k);
-- Standby (after replay of CREATE INDEX, in the same transaction):
SELECT count(*) FROM t WHERE k > 0;
-- actual: 1000
SET enable_seqscan = off;
SET enable_bitmapscan = off;
SET enable_indexscan = on;
SET enable_indexonlyscan = on;
SELECT count(*) FROM t WHERE k > 0;
-- actual: 500
-- expected: 1000 (or transaction canceled by recovery conflict)
COMMIT;
** SQL-language function plan cache freezes parse-time literal coercions **
In PostgreSQL 18 and later, non-inlined SQL-language functions cache
their parse-analyzed Query tree across calls for the lifetime of the
session. Because unknown-literal input conversion happens during parse
analysis, parse-time literal coercions inside a SQL function body are
frozen on the first call until a relation or function invalidation or a
search_path change occurs. Consequently, INSERT ... VALUES ('now', ...)
inside a SQL function writes the first call's transaction timestamp in
all subsequent transactions, and literals whose parsing depends on GUC
settings such as DateStyle or TimeZone retain the meaning from the
first call after those GUCs change. In PostgreSQL 17 and earlier, a
non-inlined SQL function body is re-parsed in every outer query, so
'now' evaluates to the current transaction's start time and GUC-
dependent literals follow current settings, matching direct query
execution. While the PL/pgSQL documentation (plpgsql.sgml, "Plan
Caching") warns about this plan-caching behavior for 'now', SQL-language
functions never had this behavior prior to 18 and neither the PostgreSQL
18 documentation nor the release notes state that SQL functions are
affected.
Reproduction:
CREATE TABLE one(x int);
INSERT INTO one VALUES (1);
CREATE FUNCTION fd() RETURNS date LANGUAGE sql AS $$SELECT
'01/02/2024'::date FROM one$$;
CREATE FUNCTION ftz() RETURNS text LANGUAGE sql AS $$SELECT
('2024-01-01 00:00'::timestamptz)::timestamp::text FROM one$$;
CREATE TABLE log(ts timestamptz, msg text);
CREATE FUNCTION log_it(m text) RETURNS void LANGUAGE sql AS $$ INSERT
INTO log VALUES ('now', m) $$;
SET TimeZone = 'UTC';
SET DateStyle = 'ISO, MDY';
SELECT fd(); -- 2024-01-02
SELECT ftz(); -- 2024-01-01 00:00:00
SELECT log_it('first');
SET DateStyle = 'ISO, DMY';
SET TimeZone = 'Asia/Tokyo';
SELECT fd(); -- actual (18): 2024-01-02 (stale
MDY parse)
-- expected (17): 2024-02-01
SELECT ftz(); -- actual (18): 2024-01-01
09:00:00 (stale UTC parse)
-- expected (17): 2024-01-01 00:00:00
SELECT '01/02/2024'::date AS fresh; -- 2024-02-01
SELECT pg_sleep(1.1);
SELECT log_it('second');
SELECT count(DISTINCT ts) AS distinct_ts FROM log;
-- actual (18): 1 (second row got the first call's 'now')
-- expected (17): 2
** ON UPDATE CASCADE into narrower-typmod FK leaves orphans or re-points rows **
When an ON UPDATE CASCADE foreign key references a primary key column
with a wider typmod (for example, varchar(n) or char(n) referencing
text, numeric(p,s) referencing numeric, or timestamp(p) referencing
timestamp), the cascade query updates the referencing column through
assignment coercion to the FK column's typmod. This coercion can alter
the cascaded value without raising an error: varchar(n) and char(n)
drop trailing blanks per the SQL length-coercion rule, and numeric(p,s)
and timestamp(p) round. As a result, the FK row either keeps its old
stored value so the FK re-check is skipped and an orphan row is
committed, or receives a rounded value that matches a different PK row
and silently re-points the reference. This violates the CREATE TABLE
documentation for ON UPDATE CASCADE ("update the values of the
referencing column(s) to the new values of the referenced column(s)")
and breaks the invariant that a validated, enforced foreign key holds
after every statement (a brute-force check and re-adding the constraint
with ALTER TABLE ADD FOREIGN KEY both fail with SQLSTATE 23503).
Reproduction:
-- A: varchar(n) FK referencing text PK: cascade value truncated
-- (trailing blanks), FK re-check skipped -> committed orphan
CREATE TABLE pv (k text PRIMARY KEY);
INSERT INTO pv VALUES ('abc');
CREATE TABLE fv (r varchar(3) REFERENCES pv ON UPDATE CASCADE);
INSERT INTO fv VALUES ('abc');
UPDATE pv SET k = 'abc ';
SELECT r, length(r) FROM fv;
-- r | length
-- -----+--------
-- abc | 3
-- (1 row)
SELECT count(*) AS orphans FROM fv WHERE NOT EXISTS (SELECT 1 FROM pv
WHERE pv.k = fv.r);
-- orphans
-- ---------
-- 1
-- (1 row)
ALTER TABLE fv ADD CONSTRAINT recheck FOREIGN KEY (r) REFERENCES pv;
-- ERROR: insert or update on table "fv" violates foreign key
constraint "recheck"
-- DETAIL: Key (r)=(abc) is not present in table "pv".
-- B: char(n) FK referencing text PK
CREATE TABLE pt (k text PRIMARY KEY);
INSERT INTO pt VALUES ('ab');
CREATE TABLE ft (r char(4) REFERENCES pt ON UPDATE CASCADE);
INSERT INTO ft VALUES ('ab');
UPDATE pt SET k = 'ab ';
ALTER TABLE ft ADD CONSTRAINT recheck FOREIGN KEY (r) REFERENCES pt;
-- ERROR: insert or update on table "ft" violates foreign key
constraint "recheck"
-- DETAIL: Key (r)=(ab ) is not present in table "pt".
-- C: numeric(5,1) FK referencing numeric PK: cascade rounds the new
-- key, FK row silently re-points to another PK row
CREATE TABLE pn (k numeric PRIMARY KEY);
INSERT INTO pn VALUES (1.2), (1.3);
CREATE TABLE fn (r numeric(5,1) REFERENCES pn ON UPDATE CASCADE);
INSERT INTO fn VALUES (1.2);
UPDATE pn SET k = 1.25 WHERE k = 1.2;
SELECT * FROM fn;
-- r
-- -----
-- 1.3
-- (1 row)
UPDATE pn SET k = 9 WHERE k = 1.25;
SELECT * FROM fn;
-- r
-- -----
-- 1.3
-- (1 row)
-- D: timestamp(0) FK referencing timestamp PK
CREATE TABLE pts (k timestamp PRIMARY KEY);
INSERT INTO pts VALUES ('2020-01-01 00:00:00'), ('2020-01-01 00:00:01');
CREATE TABLE fts (r timestamp(0) REFERENCES pts ON UPDATE CASCADE);
INSERT INTO fts VALUES ('2020-01-01 00:00:00');
UPDATE pts SET k = '2020-01-01 00:00:00.7' WHERE k = '2020-01-01 00:00:00';
SELECT * FROM fts;
-- r
-- ---------------------
-- 2020-01-01 00:00:01
-- (1 row)
Expected behavior:
After any statement, a validated, enforced foreign key constraint must
hold (0 orphans, and re-adding the same constraint with ALTER TABLE ADD
FOREIGN KEY must succeed), and cascading an update must not silently
re-point a referencing row to a different PK row. Either the cascade
must fail when the coerced FK value differs from the new PK key (or
does not equal it under the FK equality operator), or the FK check must
run on cascaded rows.
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Etsuro Fujita | 2026-11-03 02:20:01 | Re: Asynchronous MergeAppend |
| Previous Message | Fujii Masao | 2026-11-03 01:57:14 | Re: Wrong LSN in WAL decoding error messages |