Re: [PATCH] Reject ENCODING option for COPY TO FORMAT JSON

From: Ayush Tiwari <ayushtiwari(dot)slg01(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: pgsql-hackers(at)postgresql(dot)org, andrew(at)dunslane(dot)net
Subject: Re: [PATCH] Reject ENCODING option for COPY TO FORMAT JSON
Date: 2026-04-20 15:01:44
Message-ID: CAJTYsWX-jsLzxGRAb-dWnEpGYRPbDYHwce8LctVE92LiDfM2Jw@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

>
> On Mon, 20 Apr 2026 at 19:09, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>

> Seems to me the correct thing here is to make it work like the other
>> cases, ie perform pg_server_to_any(). I have exactly no sympathy for
>> the argument about the RFC saying it must be UTF-8, not least because
>> that's not in fact what is implemented (what if the server encoding
>> isn't UTF-8?).
>>
>
> Agreed. I initially thought rejecting the option was the safer route
> given the RFC, but as you pointed out, we aren't enforcing
> UTF-8 strictly on the server side anyway.
>
>
>> Rejecting this option altogether doesn't improve anything, not
>> functionally, not specs-compliance-wise, nor according to the
>> principle of least surprise.
>>
>
> Makes sense. Implementing the conversion properly
> keeps JSON format consistent with how the text and CSV formats behave.
>
>>
>> No, you don't get to punt this till later. Once we ship v19 there's
>> going to be a strong expectation of backwards compatibility.
>>
>> The idea of sending UTF-8 to a client that's set client_encoding to
>> something else would be risible, if it weren't a security hazard.
>>
>
> I agree sending unconverted bytes to a mismatched
> client encoding is clearly a security hazard that needs addressing. Did
> not consider the backward compatibility part, my bad.
>
> Was trying out adding pg_server_to_any() to the json_buf after
> composite_to_json() returns,
> correctly covering both explicit ENCODING option specifications and
> implicit client_encoding mismatches.
>
> Let me send a patch with code and associated test cases.
>
>
Attached patch with round trip test case. Please review and let me
know if it's in the right direction.

Regards,
Ayush

Attachment Content-Type Size
v3-0001-Apply-encoding-conversion-in-COPY-TO-FORMAT-JSON.patch application/octet-stream 4.6 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2026-04-20 15:11:56 Re: Adding REPACK [concurrently]
Previous Message Greg Burd 2026-04-20 14:55:09 Re: Add bms_offset_members() function for bitshifting Bitmapsets