| From: | jian he <jian(dot)universality(at)gmail(dot)com> |
|---|---|
| To: | Andrew Dunstan <andrew(at)dunslane(dot)net> |
| Cc: | Joe Conway <mail(at)joeconway(dot)com>, Junwang Zhao <zhjwpku(at)gmail(dot)com>, Florents Tselai <florents(dot)tselai(at)gmail(dot)com>, "Andrey M(dot) Borodin" <x4mmm(at)yandex-team(dot)ru>, Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>, Daniel Verite <daniel(at)manitou-mail(dot)org>, Davin Shearer <davin(at)apache(dot)org>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
| Subject: | Re: Emitting JSON to file using COPY TO |
| Date: | 2026-03-09 03:48:26 |
| Message-ID: | CACJufxGm3OhbG3=bj4nLxMzq92A84v6Q1dG+hR_ZFrfLg1B99w@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-general pgsql-hackers |
On Mon, Mar 9, 2026 at 3:44 AM Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
>
> Hmm. But should we be scribbling on slot->tts_tupleDescriptor like that?
> How about something like this?:
>
> - * Full table or query without column list. Ensure the slot uses
> - * cstate->tupDesc so that the datum is stamped with the right type;
> - * for queries output type is RECORDOID this must be the blessed
> - * descriptor so that composite_to_json can look it up via
> - * lookup_rowtype_tupdesc.
> + * Full table or query without column list. For queries, the slot's
> + * TupleDesc may carry RECORDOID, which is not registered in the
> type
> + * cache and would cause composite_to_json's lookup_rowtype_tupdesc
> + * call to fail. Build a HeapTuple stamped with the blessed
> + * descriptor so the type can be looked up correctly.
> */
> if (!cstate->rel && slot->tts_tupleDescriptor->tdtypeid ==
> RECORDOID)
> - slot->tts_tupleDescriptor = cstate->queryDesc->tupDesc;
> + {
> + HeapTuple tup;
>
> - rowdata = ExecFetchSlotHeapTupleDatum(slot);
> + tup = heap_form_tuple(cstate->tupDesc,
> + slot->tts_values,
> + slot->tts_isnull);
> + rowdata = HeapTupleGetDatum(tup);
> + }
> + else
> + {
> + rowdata = ExecFetchSlotHeapTupleDatum(slot);
> + }
>
This is better. I've tried to get rid of json_projvalues and json_projnulls.
Just using heap_form_tuple, but it won't work.
I incorporated the v28-0004 COPY column list into v9-0002.
With this patch set, we added four fields to the struct CopyToStateData.
+ StringInfo json_buf; /* reusable buffer for JSON output,
+ * initialized in BeginCopyTo */
+ TupleDesc tupDesc; /* Descriptor for JSON output; for a column
+ * list this is a projected descriptor */
+ Datum *json_projvalues; /* pre-allocated projection values, or
+ * NULL */
+ bool *json_projnulls; /* pre-allocated projection nulls, or NULL */
Using the script in
https://www.postgresql.org/message-id/CACJufxFFZqxC3p4WjpTEi4riaJm%3DpADX%2Bpy0yQ0%3DRWTn5cqK3Q%40mail.gmail.com
I tested it again on macOS and Linux, and there are no regressions for
COPY TO with the TEXT and CSV formats.
| Attachment | Content-Type | Size |
|---|---|---|
| v29-0002-json-format-for-COPY-TO.patch | text/x-patch | 29.3 KB |
| v29-0001-introduce-CopyFormat-refactor-CopyFormatOptions.patch | text/x-patch | 13.1 KB |
| v29-0003-Add-option-force_array-for-COPY-JSON-FORMAT.patch | text/x-patch | 12.3 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Christoph Moench-Tegeder | 2026-03-09 07:18:25 | Re: CREATE TABLE fails |
| Previous Message | Adrian Klaver | 2026-03-09 01:22:12 | Re: CREATE TABLE fails |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Richard Guo | 2026-03-09 04:01:14 | Re: Convert NOT IN sublinks to anti-joins when safe |
| Previous Message | Ashutosh Bapat | 2026-03-09 03:47:21 | Re: Options to control remote transactions’ access/deferrable modes in postgres_fdw |