Re: Emitting JSON to file using COPY TO

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Joe Conway <mail(at)joeconway(dot)com>
Cc: Nathan Bossart <nathandbossart(at)gmail(dot)com>, Davin Shearer <davin(at)apache(dot)org>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Emitting JSON to file using COPY TO
Date: 2023-12-06 21:03:12
Message-ID: 8724f1d2-d11c-aaf2-2cd3-3725e9441d1e@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers


On 2023-12-06 We 15:20, Tom Lane wrote:
> Joe Conway <mail(at)joeconway(dot)com> writes:
>> I'll see if I can add some caching to composite_to_json(), but based on
>> the relative data size it does not sound like there is much performance
>> left on the table to go after, no?
> If Nathan's perf results hold up elsewhere, it seems like some
> micro-optimization around the text-pushing (appendStringInfoString)
> might be more useful than caching. The 7% spent in cache lookups
> could be worth going after later, but it's not the top of the list.
>
> The output size difference does say that maybe we should pay some
> attention to the nearby request to not always label every field.
> Perhaps there should be an option for each row to transform to
> a JSON array rather than an object?
>
>

I doubt it. People who want this are likely to want pretty much what
this patch is providing, not something they would have to transform in
order to get it. If they want space-efficient data they won't really be
wanting JSON. Maybe they want Protocol Buffers or something in that vein.

I see there's  nearby proposal to make this area pluggable at
<https://postgr.es/m/20231204.153548.2126325458835528809.kou@clear-code.com>

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Joe Conway 2023-12-06 21:28:59 Re: Emitting JSON to file using COPY TO
Previous Message Tom Lane 2023-12-06 20:20:46 Re: Emitting JSON to file using COPY TO

Browse pgsql-hackers by date

  From Date Subject
Next Message Matthias van de Meent 2023-12-06 21:08:38 Reducing output size of nodeToString
Previous Message Andrew Dunstan 2023-12-06 20:52:10 Re: Remove MSVC scripts from the tree