Re: [HACKERS] Logical replication and multimaster

From: Jon Erdman <jon(at)thewickedtribe(dot)net>
To: Jim Nasby <Jim(dot)Nasby(at)BlueTreble(dot)com>, David Fetter <david(at)fetter(dot)org>, Craig Ringer <craig(at)2ndquadrant(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Simon Riggs <simon(at)2ndquadrant(dot)com>, Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Jon Erdman <jon(at)thewickedtribe(dot)net>
Subject: Re: [HACKERS] Logical replication and multimaster
Date: 2023-10-05 05:31:00
Message-ID: 0101018afe515ec9-a3b31eeb-8040-4c9d-a8e3-02cf0c7ed197-000000@us-west-2.amazonses.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


Oops,

Looking at my old message from 2015 on this subject, it was Magnus who
pish-poshed it, and Page who liked it. Don't want to throw shaded at the
wrong person ;)
--
Jon Erdman (aka StuckMojo)
PostgreSQL Zealot

On 10/5/23 12:14 AM, Jon Erdman wrote:
>
> Well,
>
> Given my earlier message about Apple wanting to pay me for writing
> patches now, maybe I can revisit this idea.
>
> For background: I brought up the idea of an FDW that could read custom
> dump files and expose them as tables so that you could grab just a
> single record (or of course more) from a custom dump file without having
> to load the whole thing up, if you were stuck reaching into a backup to
> get at accidentally deleted tables, rows, etc.. The stopper, which was
> communicated to me by Tom at the following pgcon was that the code for
> parsing custom dumps is duplicated in pg_dump only, and would need to be
> duplicated into the server for the FDW, or broken out into a library.
>
> And for posterity, Dave Page said that was a stupid idea, while Magnus
> said that it sounded useful. And IIRC Bruce and Robert H said it was
> doable, just a good deal of work on the refactor needed.
>
> This convo went down at the Amsterdam conf where I spoke about using
> writeable LVM snapshots to expose each developer a copy of prod to
> noodle on, without having to actually make a full copy for each dev.
>
> Added trivia: I gave the talk with a can of Heineken in my hand at the
> podium, and my lightning talk had the work F(&king Cool in the title ;)
>
> That was also when I bought my plush Slony which was supposedly the very
> last one. (turns out they made more)
> --
> Jon Erdman (aka StuckMojo)
>     PostgreSQL Zealot
>
> On 12/15/15 9:48 PM, Jim Nasby wrote:
>> On 12/13/15 7:37 AM, David Fetter wrote:
>>> As I understand it, pushing these into a library has been proposed but
>>> not rejected.  That it hasn't happened yet is mostly about the lack of
>>> tuits (the round ones) to rewrite the functionality as libraries and
>>> refactor pg_dump/pg_restore to use only calls to same.  As usual, it's
>>> less about writing the code and more about the enormous amount of
>>> testing any such a refactor would entail.
>>
>> My understanding as well. IIRC Jon Erdman brought this question up a
>> couple years ago and the response was "It'd probably be accepted, it's
>> just that no one has done the work."
>>
>>> I believe that refactoring much of pg_dump's functionality for the
>>> current version of the server into SQL-accessible functions and making
>>> pg_dump use only those functions is achievable with available
>>> resources.
>>>
>>> Such a refactor need not be all-or-nothing.  For example, the
>>> dependency resolution stuff is a first step that appears to be worth
>>> doing by itself even if the effort then pauses, possibly for some
>>> time.
>>
>> If someone wanted to spend time on this, I suspect it'd be worth
>> looking at how bad some of the backward compatibility issues would be
>> if done in the server. Maybe they wouldn't be that bad. I suspect the
>> audience for this code would be much larger if it was in the server as
>> opposed to a C library.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Gurjeet Singh 2023-10-05 05:41:15 Re: [PoC/RFC] Multiple passwords, interval expirations
Previous Message Michael Paquier 2023-10-05 05:23:36 Re: Making aggregate deserialization (and WAL receive) functions slightly faster