Re: backup manifests

From: David Steele <david(at)pgmasters(dot)net>
To: Stephen Frost <sfrost(at)snowman(dot)net>, David Fetter <david(at)fetter(dot)org>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Robert Haas <robertmhaas(at)gmail(dot)com>, Tels <nospam-pg-abuse(at)bloodgate(dot)com>, Suraj Kharage <suraj(dot)kharage(at)enterprisedb(dot)com>, Rushabh Lathia <rushabh(dot)lathia(at)gmail(dot)com>, Andrew Dunstan <andrew(dot)dunstan(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Jeevan Chalke <jeevan(dot)chalke(at)enterprisedb(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>
Subject: Re: backup manifests
Date: 2020-01-15 04:36:30
Message-ID: 1115c3d4-2d80-8388-4fd4-5fdb3ee9b429@pgmasters.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Stephen,

On 1/14/20 1:35 PM, Stephen Frost wrote:
>
> My thought, which I had expressed to David (though he obviously didn't
> entirely agree with me since he suggested the other options), was to
> adapt the pgBackRest JSON parser, which isn't really all that much code.

It's not that I didn't agree, it's just that the pgBackRest code does
use mem contexts, the type system, etc. After looking at some other
solutions with similar amounts of code I thought they might be more
acceptable. At least it seemed like a good idea to throw it out there.

> Even so, David's offered to adjust the code to use the frontend's memory
> management (*cough* malloc()..), and error handling/logging, and he had
> some idea for Variadics (or maybe just pulling the backend's Datum
> system in..? He could answer better), and basically write a frontend
> JSON parser for PG without too much code, no external dependencies, and
> to make sure it answers this requirement, and I've agreed that he can
> spend some time on that instead of pgBackRest to get us through this, if
> everyone else is agreeable to the idea.

To keep it simple I think we are left with callbacks or a somewhat
static "what's the next datum" kind of approach. I think the latter
could get us through a release or two while we make improvements.

> Obviously this isn't intended
> to box anyone in- if there turns out even after the code's been written
> to be some fatal issue with using it, so be it, but we're offering to
> help.

I'm happy to work up a prototype unless the consensus is that we
absolutely don't want a second JSON parser in core.

Regards,
--
-David
david(at)pgmasters(dot)net

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2020-01-15 04:47:13 Re: backup manifests
Previous Message Masahiko Sawada 2020-01-15 04:34:33 Re: [HACKERS] Block level parallel vacuum