pgsql: Apply encoding conversion in COPY TO FORMAT JSON

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Apply encoding conversion in COPY TO FORMAT JSON
Date: 2026-05-30 01:59:04
Message-ID: E1wT8ye-000CaS-0r@gemulon.postgresql.org
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Apply encoding conversion in COPY TO FORMAT JSON

CopyToJsonOneRow() sent the output of composite_to_json() directly
via CopySendData() without encoding conversion. The text and CSV
paths convert per-attribute via pg_server_to_any() when
need_transcoding is true, but the JSON path skipped this entirely.

This meant COPY ... TO ... WITH (FORMAT json, ENCODING 'LATIN1') on
a UTF-8 server silently produced UTF-8 output, and COPY TO STDOUT
with a non-UTF-8 client_encoding would send unconverted bytes to
the client.

Apply pg_server_to_any() to the whole JSON buffer after
composite_to_json() returns, converting to the requested file
encoding when it differs from the server encoding. Tests cover
both the explicit ENCODING option and the implicit case where
file_encoding is inherited from client_encoding.

Introduced by 7dadd38cda9 (json format for COPY TO).

Author: Ayush Tiwari <ayushtiwari(dot)slg01(at)gmail(dot)com>
Reviewed-by: Andrew Dunstan <andrew(at)dunslane(dot)net>
Discussion: https://postgr.es/m/CAJTYsWX-jsLzxGRAb-dWnEpGYRPbDYHwce8LctVE92LiDfM2Jw@mail.gmail.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/7dc5bbcf220be05786a08aad2455c61b6dbd4b78

Modified Files
--------------
src/backend/commands/copyto.c | 20 +++++++++++++++++++-
src/test/regress/expected/copyencoding.out | 27 +++++++++++++++++++++++++++
src/test/regress/sql/copyencoding.sql | 19 +++++++++++++++++++
3 files changed, 65 insertions(+), 1 deletion(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Bruce Momjian 2026-05-30 18:55:54 pgsql: doc PG 19 relnotes: adjust pg_read_all_data & pg_write_all_data
Previous Message Álvaro Herrera 2026-05-29 22:23:55 pgsql: Allow old WAL recycling during REPACK CONCURRENTLY