Re: Emitting JSON to file using COPY TO

From: Joe Conway <mail(at)joeconway(dot)com>
To: "David G(dot) Johnston" <david(dot)g(dot)johnston(at)gmail(dot)com>
Cc: Daniel Verite <daniel(at)manitou-mail(dot)org>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Davin Shearer <davin(at)apache(dot)org>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Emitting JSON to file using COPY TO
Date: 2023-12-07 00:57:42
Message-ID: 63119a48-07aa-4cab-9383-91a24b7b54cd@joeconway.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On 12/6/23 19:39, David G. Johnston wrote:
> On Wed, Dec 6, 2023 at 4:45 PM Joe Conway <mail(at)joeconway(dot)com
> <mailto:mail(at)joeconway(dot)com>> wrote:
>
>
> " The backend sends a CopyOutResponse message to the frontend, followed
>     by zero or more CopyData messages (always one per row), followed by
>     CopyDone"
>
> probably "always one per row" would be changed to note that json array
> format outputs two extra rows for the start/end bracket.
>
>
> Fair, I was ascribing much more semantic meaning to this than it wants.
>
> I don't see any real requirement, given the lack of semantics, to
> mention JSON at all.  It is one CopyData per row, regardless of the
> contents.  We don't delineate between the header and non-header data in
> CSV.  It isn't a protocol concern.

good point

> But I still cannot shake the belief that using a format code of 1 -
> which really could be interpreted as meaning "textual csv" in practice -
> for this JSON output is unwise and we should introduce a new integer
> value for the new fundamental output format.

No, I am pretty sure you still have that wrong. The "1" means binary
mode. As in
8<----------------------
FORMAT

Selects the data format to be read or written: text, csv (Comma
Separated Values), or binary. The default is text.
8<----------------------

That is completely separate from text and csv. It literally means to use
the binary output functions instead of the usual ones:

8<----------------------
if (cstate->opts.binary)
getTypeBinaryOutputInfo(attr->atttypid,
&out_func_oid,
&isvarlena);
else
getTypeOutputInfo(attr->atttypid,
&out_func_oid,
&isvarlena);
8<----------------------

Both "text" and "csv" mode use are non-binary output formats. I believe
the JSON output format is also non-binary.

--
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message David G. Johnston 2023-12-07 01:09:22 Re: Emitting JSON to file using COPY TO
Previous Message David G. Johnston 2023-12-07 00:39:11 Re: Emitting JSON to file using COPY TO

Browse pgsql-hackers by date

  From Date Subject
Next Message David G. Johnston 2023-12-07 01:09:22 Re: Emitting JSON to file using COPY TO
Previous Message Michael Paquier 2023-12-07 00:51:28 Re: Is WAL_DEBUG related code still relevant today?