Re: Emitting JSON to file using COPY TO

From: Davin Shearer <davin(at)apache(dot)org>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Cc: Joe Conway <mail(at)joeconway(dot)com>
Subject: Re: Emitting JSON to file using COPY TO
Date: 2023-12-04 22:55:00
Message-ID: CALvfUkDxgTppjy-Q_i2znakMFp2vrJdPUEJNyGTL2LcZBOWV6A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

Sorry about the top posting / top quoting... the link you sent me gives me
a 404. I'm not exactly sure what top quoting / posting means and Googling
those terms wasn't helpful for me, but I've removed the quoting that my
mail client is automatically "helpfully" adding to my emails. I mean no
offense.

Okay, digging in more...

If the value contains text that has BOMs [footnote 1] in it, it must be
preserved (the database doesn't need to interpret them or do anything
special with them - just store it and fetch it). There are however a few
characters that need to be escaped (per
https://www.w3docs.com/snippets/java/how-should-i-escape-strings-in-json.html)
so that the JSON format isn't broken. They are:

1. " (double quote)
2. \ (backslash)
3. / (forward slash)
4. \b (backspace)
5. \f (form feed)
6. \n (new line)
7. \r (carriage return)
8. \t (horizontal tab)

These characters should be represented in the test cases to see how the
escaping behaves and to ensure that the escaping is done properly per JSON
requirements. Forward slash comes as a bit of a surprise to me, but `jq`
handles it either way:

➜ echo '{"key": "this / is a forward slash"}' | jq .
{
"key": "this / is a forward slash"
}
➜ echo '{"key": "this \/ is a forward slash"}' | jq .
{
"key": "this / is a forward slash"
}

Hope it helps, and thank you!

1. I don't disagree that BOMs shouldn't be used for UTF-8, but I'm also
processing UTF-16{BE,LE} and UTF-32{BE,LE} (as well as other textural
formats that are neither ASCII or Unicode). I don't have the luxury of
changing the data that is given.

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2023-12-05 00:43:07 Re: Help understand why DELETE is so slow
Previous Message Merlin Moncure 2023-12-04 21:51:59 Re: Help understand why DELETE is so slow

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2023-12-04 23:45:03 Re: PATCH: Add REINDEX tag to event triggers
Previous Message Michael Paquier 2023-12-04 22:29:59 Re: Remove MSVC scripts from the tree