Re: Emitting JSON to file using COPY TO

From: Joe Conway <mail(at)joeconway(dot)com>
To: Davin Shearer <davin(at)apache(dot)org>, PostgreSQL-development <pgsql-hackers(at)postgreSQL(dot)org>
Subject: Re: Emitting JSON to file using COPY TO
Date: 2023-12-01 19:28:55
Message-ID: 24e3ee88-ec1e-421b-89ae-8a47ee0d2df1@joeconway.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On 11/29/23 10:32, Davin Shearer wrote:
> Thanks for the responses everyone.
>
> I worked around the issue using the `psql -tc` method as Filip described.
>
> I think it would be great to support writing JSON using COPY TO at
> some point so I can emit JSON to files using a PostgreSQL function directly.
>
> -Davin
>
> On Tue, Nov 28, 2023 at 2:36 AM Filip Sedlák <filip(at)sedlakovi(dot)org
> <mailto:filip(at)sedlakovi(dot)org>> wrote:
>
> This would be a very special case for COPY. It applies only to a single
> column of JSON values. The original problem can be solved with psql
> --tuples-only as David wrote earlier.
>
>
> $ psql -tc 'select json_agg(row_to_json(t))
>               from (select * from public.tbl_json_test) t;'
>
>   [{"id":1,"t_test":"here's a \"string\""}]
>
>
> Special-casing any encoding/escaping scheme leads to bugs and harder
> parsing.

(moved to hackers)

I did a quick PoC patch (attached) -- if there interest and no hard
objections I would like to get it up to speed for the January commitfest.

Currently the patch lacks documentation and regression test support.

Questions:
----------
1. Is supporting JSON array format sufficient, or does it need to
support some other options? How flexible does the support scheme need to be?

2. This only supports COPY TO and we would undoubtedly want to support
COPY FROM for JSON as well, but is that required from the start?

Thanks for any feedback.

--
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachment Content-Type Size
copyto_json.000.diff text/x-patch 8.3 KB

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Nathan Bossart 2023-12-01 23:09:58 Re: Emitting JSON to file using COPY TO
Previous Message Owen Nelson 2023-12-01 19:08:49 Re: Understanding partial index selection

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2023-12-01 19:49:04 Re: Building PosgresSQL with LLVM fails on Solaris 11.4
Previous Message shihao zhong 2023-12-01 19:18:29 Re: Fix bogus Asserts in calc_non_nestloop_required_outer