Re: GDQ iimplementation (was: Re: Clustering features for upcoming developer meeting -- please claim yours!)

From: Simon Riggs <simon(at)2ndQuadrant(dot)com>
To: Jan Wieck <JanWieck(at)Yahoo(dot)com>
Cc: Marko Kreen <markokr(at)gmail(dot)com>, Hannu Krosing <hannu(at)2ndquadrant(dot)com>, Josh Berkus <josh(at)agliodbs(dot)com>, pgsql-cluster-hackers(at)postgresql(dot)org
Subject: Re: GDQ iimplementation (was: Re: Clustering features for upcoming developer meeting -- please claim yours!)
Date: 2010-05-11 13:19:04
Message-ID: 1273583944.308.173.camel@ebony
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-cluster-hackers

On Tue, 2010-05-11 at 08:33 -0400, Jan Wieck wrote:

> What are the advantages of anything proposed over the current
> implementations used by Londiste and Slony?

It would be good to have a core technology that provided a generic
transport to other remote databases.

We already have WALSender and WALReceiver, which uses the COPY protocol
as a transport mechanism. It would be easy to extend that so we could
send other forms of data.

We can do that in two ways:

* Alter triggers so that Slony/Londiste write directly to WAL rather
than log tables, using a new WAL record for custom data blobs.

* Alter WALSender so it can read Slony/Londiste log tables for
consumption by an architecture similar to WALreceiver/Startup. Probably
easier.

We can also alter the WAL format itself to include the information in
WAL that is required to do what Slony/Londiste already do, so we don't
need to specifically write anything at all, just read WAL at other end.
Even more efficient.

The advantages of these options would be

* integration of core technologies
* greater efficiency for trigger based logging via WAL

In other RDBMS "replication" has long meant "data transport, either for
HA or application use". We should be looking beyond the pure HA aspects,
as pgq does.

I would certainly like to see a system that wrote data on master and
then constructed the SQL on receiver-side (i.e. on slave), so the
integration was less tight. That would allow data to be sent and for it
to be consumed to a variety of purposes, not just HA replay.

--
Simon Riggs www.2ndQuadrant.com

In response to

Responses

Browse pgsql-cluster-hackers by date

  From Date Subject
Next Message Marko Kreen 2010-05-11 13:36:26 Re: GDQ iimplementation (was: Re: Clustering features for upcoming developer meeting -- please claim yours!)
Previous Message Jan Wieck 2010-05-11 12:33:12 GDQ iimplementation (was: Re: Clustering features for upcoming developer meeting -- please claim yours!)