Re: Non-text mode for pg_dumpall

On 2025-07-29 Tu 4:09 PM, Andrew Dunstan wrote:
>
> On 2025-07-28 Mo 8:04 AM, Andrew Dunstan wrote:
>>
>> On 2025-07-27 Su 7:56 PM, Noah Misch wrote:
>>> On Fri, Jul 25, 2025 at 04:59:29PM -0400, Tom Lane wrote:
>>>> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>>>>> Before we throw the baby out with the bathwater, how about this
>>>>> suggestion? pg_dumpall would continue to produce globals.dat, but it
>>>>> wouldn't be processed by pg_restore, which would only restore the
>>>>> individual databases. Or else we just don't produce globals.dat at
>>>>> all.
>>>>> Then we could introduce a structured object that pg_restore could
>>>>> safely
>>>>> use for release 19, and I think we'd still have something useful for
>>>>> release 18.
>>>> I dunno ... that seems like a pretty weird behavior.  People would
>>>> have to do a separate text-mode "pg_dumpall -g" and remember to
>>>> restore that too.  Admittedly, this could be more convenient than
>>>> "pg_dumpall -g" plus separately pg_dump'ing each database, which is
>>>> what people have to do today if they want anything smarter than a flat
>>>> text dumpfile.  But it still seems like a hack --- and it would not be
>>>> compatible with v19, where presumably "pg_dumpall | pg_restore"
>>>> *would* restore globals.  I think that the prospect of changing
>>>> dump/restore scripts and then having to change them again in v19
>>>> isn't too appetizing.
>>> +1
>>
>>
>> OK, got it. Will revert.
>>
>>
>>
>
> here's a reversion patch for master. It applies cleanly to release 18
> as well. Thanks to Mahendra Singh Thalor for helping me sanity check
> it (Any issues are of course my responsibility)
>
>
> I'll work on pulling the entry out of the release notes.
>
>
>

OK, now that's reverted we should discuss how to proceed. I had two
thoughts - we could use invent a JSON format for the globals, or we
could just use the existing archive format. I think the archive format
is pretty flexible, and should be able to accommodate this. The downside
is it's not humanly readable. The upside is that we don't need to do
anything special either to write it or parse it.

There might also be other reasonable options. But I think we should stay
out of the business of using custom code to parse text.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com