Fix DEFAULT marker handling in COPY FROM ... HEADER MATCH

From: Ayoub Kazar <ayoub(dot)kazar(at)data-bene(dot)io>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Fix DEFAULT marker handling in COPY FROM ... HEADER MATCH
Date: 2026-11-04 00:21:36
Message-ID: 2da457c8-a9a1-49b1-a89f-719190dff7ae@data-bene.io
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

I noticed an issue when combining the HEADER MATCH option (added in v15)
with the DEFAULT option in COPY FROM (added in v16).

If any column name in the header line matches the configured DEFAULT marker,
two problematic behaviors occur depending on whether the column has a
default:

-When the column has a default value, the input value of the first data row
 is discarded and replaced with the column's default expression. All
 subsequent data rows are loaded normally:

 CREATE TABLE sales (id int, "default" int DEFAULT 0, note text);

 COPY sales FROM STDIN WITH (FORMAT csv, HEADER MATCH, DEFAULT 'default');
 id,default,note
 1,5,first
 2,7,second
 \.

 SELECT * FROM sales ORDER BY id;
 id | default |  note
 ----+---------+--------
 1 |       0 | first    -- Expected 5 here ?
 2 |       7 | second
 (2 rows)

-When the column does not have a default value, COPY aborts during header
 parsing, but not for the correct reason i guess:

 CREATE TABLE t (d int, b text);
COPY t FROM STDIN WITH (FORMAT csv, HEADER MATCH, DEFAULT 'd');
 d,b
 5,x
 \.
 ERROR:  unexpected default marker in COPY data
 DETAIL:  Column "d" has no default value.
 CONTEXT:  COPY t, line 1: "d,b"

This also happens with foreign tables via file_fdw when used with
OPTIONS (header 'match', default '...').

Because header parsing and the first row of data are parsed within the same
first call to NextCopyFrom(), and cstate->defaults is initialized to false
only at the top of NextCopyFrom(), the flag set while reading the header
line survived into the first data row. CopyFromTextLikeOneRow() then sees
cstate->defaults[m] set to true and evaluate the default expression
instead of
the supplied field.

The attached fix temporarily disables default marker recognition while
parsing
the header line when HEADER MATCH. I also added two tests for this.

This affects REL_16_STABLE through master.

Thoughts?

Regards,
Ayoub Kazar

Attachment Content-Type Size
0001-Fix-DEFAULT-marker-handling-in-COPY-FROM-.-HEADER-MATCH.patch text/x-patch 4.7 KB

Browse pgsql-hackers by date

  From Date Subject
Previous Message Michael Paquier 2026-11-03 23:45:16 Re: 007_pre_auth.pl fails if injection_points_wakeup hits unintended process