| From: | Andrew Dunstan <andrew(at)dunslane(dot)net> |
|---|---|
| To: | jian he <jian(dot)universality(at)gmail(dot)com> |
| Cc: | Joe Conway <mail(at)joeconway(dot)com>, Junwang Zhao <zhjwpku(at)gmail(dot)com>, Florents Tselai <florents(dot)tselai(at)gmail(dot)com>, "Andrey M(dot) Borodin" <x4mmm(at)yandex-team(dot)ru>, Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>, Daniel Verite <daniel(at)manitou-mail(dot)org>, Davin Shearer <davin(at)apache(dot)org>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
| Subject: | Re: Emitting JSON to file using COPY TO |
| Date: | 2026-03-06 12:33:31 |
| Message-ID: | 57db4b6e-51ba-4984-9343-bc62a3ee0b7c@dunslane.net |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-general pgsql-hackers |
On 2026-03-06 Fr 4:38 AM, jian he wrote:
> COPY (SELECT 1 UNION ALL SELECT 2) TO stdout WITH (format json);
>
> still fails with v25-0002, json_tupledesc_ready is not helpful.
> I think I figured it out. We need to use BlessTupleDesc in BeginCopyTo.
> Then let slot->tts_tupleDescriptor point to cstate->queryDesc->tupDesc
> in CopyToJsonOneRow
>
> * CSV, text and json formats share the same TextLike routines except for the
> * one-row callback.
> This comment is not useful, I want to delete it.
>
> CopyToTextLikeStart
> + /* JSON-specific initialization */
> + if (cstate->opts.format == COPY_FORMAT_JSON)
> + {
> + MemoryContext oldcxt;
> +
> + /* Allocate reusable JSON output buffer in long-lived context */
> + oldcxt = MemoryContextSwitchTo(cstate->copycontext);
> + initStringInfo(&cstate->json_buf);
> + MemoryContextSwitchTo(oldcxt);
> + }
> We ca just add
>
> cstate->json_buf = makeStringInfo();
>
> in BeginCopyTo.
>
> v25-0004-COPY-TO-JSON-build-JSON-per-column-support-colum.patch
> added several fields to the CopyToStateData.
> Actually, there is a simpler way (construct a new Tupdesc and let
> composite_to_json do the job), please see my v26-0004.
>
Yeah, I got some crashes using this, but reworked it a bit, and all now
seems good. I was apparently mistaken about the supposed speedup, but
the performance is steady.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
| Attachment | Content-Type | Size |
|---|---|---|
| v27-0001-introduce-CopyFormat-refactor-CopyFormatOptions.patch | text/x-patch | 13.1 KB |
| v27-0002-json-format-for-COPY-TO.patch | text/x-patch | 23.3 KB |
| v27-0003-Add-option-force_array-for-COPY-JSON-FORMAT.patch | text/x-patch | 12.3 KB |
| v27-0004-COPY-TO-JSON-support-column-lists.patch | text/x-patch | 12.5 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Adrian Klaver | 2026-03-06 16:03:07 | Re: Index (primary key) corrupt? |
| Previous Message | Wim Rouquart | 2026-03-06 10:31:04 | RE: Index (primary key) corrupt? |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Andrei Lepikhov | 2026-03-06 12:55:23 | Re: Skipping schema changes in publication |
| Previous Message | Jonathan Gonzalez V. | 2026-03-06 12:10:12 | Re: pg_upgrade fails when extension_control_path is used |