Re: Fix DEFAULT marker handling in COPY FROM ... HEADER MATCH

> On Nov 4, 2026, at 08:21, Ayoub Kazar <ayoub(dot)kazar(at)data-bene(dot)io> wrote:
>
> Hello,
>
> I noticed an issue when combining the HEADER MATCH option (added in v15) with the DEFAULT option in COPY FROM (added in v16).
>
> If any column name in the header line matches the configured DEFAULT marker,
> two problematic behaviors occur depending on whether the column has a default:
>
> -When the column has a default value, the input value of the first data row
> is discarded and replaced with the column's default expression. All
> subsequent data rows are loaded normally:
>
> CREATE TABLE sales (id int, "default" int DEFAULT 0, note text);
>
> COPY sales FROM STDIN WITH (FORMAT csv, HEADER MATCH, DEFAULT 'default');
> id,default,note
> 1,5,first
> 2,7,second
> \.
>
> SELECT * FROM sales ORDER BY id;
> id | default | note
> ----+---------+--------
> 1 | 0 | first -- Expected 5 here ?
> 2 | 7 | second
> (2 rows)
>
> -When the column does not have a default value, COPY aborts during header
> parsing, but not for the correct reason i guess:
>
> CREATE TABLE t (d int, b text);
> COPY t FROM STDIN WITH (FORMAT csv, HEADER MATCH, DEFAULT 'd');
> d,b
> 5,x
> \.
> ERROR: unexpected default marker in COPY data
> DETAIL: Column "d" has no default value.
> CONTEXT: COPY t, line 1: "d,b"
>
> This also happens with foreign tables via file_fdw when used with
> OPTIONS (header 'match', default '...').
>
>
> Because header parsing and the first row of data are parsed within the same
> first call to NextCopyFrom(), and cstate->defaults is initialized to false
> only at the top of NextCopyFrom(), the flag set while reading the header
> line survived into the first data row. CopyFromTextLikeOneRow() then sees
> cstate->defaults[m] set to true and evaluate the default expression instead of
> the supplied field.
>
> The attached fix temporarily disables default marker recognition while parsing
> the header line when HEADER MATCH. I also added two tests for this.
>
>
> This affects REL_16_STABLE through master.
>
> Thoughts?
>
>
> Regards,
> Ayoub Kazar<0001-Fix-DEFAULT-marker-handling-in-COPY-FROM-.-HEADER-MATCH.patch>

Nice catch. I reproduced the problem and confirmed that the v1 fix works.

The root cause is that CopyReadAttributesCSV() and CopyReadAttributesText() use the same logic to parse header lines and data lines, so a column name may be incorrectly interpreted as a DEFAULT marker. From this perspective, I think adding an is_header parameter to these two functions to distinguish header parsing from data parsing would be a more general solution.

With your finding, I realized that NULL markers have the same problem:
```
evantest=# create table t (id int, a int);
CREATE TABLE
evantest=# copy t from stdin with (format csv, header match, null 'a');
Enter data to be copied followed by a newline.
End with a backslash and a period on a line by itself, or an EOF signal.
>> id,a
>> 1,2
>> \.
ERROR: column name mismatch in header line field 2: got null value ("a"), expected "a"
CONTEXT: COPY t, line 1: "id,a"
```

Here, a is a valid column name in the header line and should not be rejected simply because it matches the NULL marker.

Adding an is_header parameter allows us to disable both NULL and DEFAULT marker recognition when parsing header lines, addressing both issues with the same approach. It also avoids temporarily modifying the COPY options.

So, I’m proposing v2 with this approach. I kept the regression tests you added in v1 and added tests for NULL markers and the text format. Would you please review v2?

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/