[BUG] psql: Make \copy from 'text' and 'csv' formats fail on NUL bytes

From: "Joel Jacobson" <joel(at)compiler(dot)org>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: [BUG] psql: Make \copy from 'text' and 'csv' formats fail on NUL bytes
Date: 2024-11-10 21:26:20
Message-ID: 5eabc9e4-509c-4f83-92c8-36d298cf930f@app.fastmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers,

When using \copy from in psql to import files containing NUL bytes (\0) in
'text' or 'csv' format, the NUL bytes were not detected and did not result in
an error, leading to silent data corruption.

This behavior is inconsistent with server-side COPY FROM, which reports an error
upon encountering NUL bytes in the 'text' and 'csv' formats.

Fix by adjusting handleCopyIn() to use the binary code path also when the copy
source is a file (i.e., copystream != pset.cur_cmd_source), even in textual
copies. This ensures that NUL bytes are detected and reported in the same way
as server-side COPY.

Example:

% printf 'test_col\nfoo\n\x00\nbar\nbaz\x00aar\nbizarre\n' > nul_bytes.data
% cat -v nul_bytes.data
test_col
foo
^@
bar
baz^(at)aar
bizarre

% psql
create table nul_bytes_test (test_col text);

--
-- HEAD
--
\copy nul_bytes_test (test_col) from 'nul_bytes.data' (format csv, header match);
COPY 3
select * from nul_bytes_test;
test_col
------------
foo
bar
bazbizarre
(3 rows)

\copy nul_bytes_test (test_col) from 'nul_bytes.data' (format text, header match);
select * from nul_bytes_test;
test_col
------------
foo
bar
bazbizarre
(3 rows)

--
-- 0001-psql-Make-copy-from-text-and-csv-formats-fail-on-NUL.patch
--
\copy nul_bytes_test (test_col) from 'nul_bytes.data' (format csv, header match);
ERROR: invalid byte sequence for encoding "UTF8": 0x00
CONTEXT: COPY nul_bytes_test, line 3
\copy nul_bytes_test (test_col) from 'nul_bytes.data' (format text, header match);
ERROR: invalid byte sequence for encoding "UTF8": 0x00
CONTEXT: COPY nul_bytes_test, line 3

/Joel

Attachment Content-Type Size
0001-psql-Make-copy-from-text-and-csv-formats-fail-on-NUL.patch application/octet-stream 1.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Heikki Linnakangas 2024-11-10 21:30:13 Re: Interrupts vs signals
Previous Message Alena Rybakina 2024-11-10 20:09:41 Re: Vacuum statistics