Re: [RFC][PATCH] wal decoding, attempt #2 - Design Documents (really attached)

On 10/15/2012 3:25 PM, Andres Freund wrote:
> On Monday, October 15, 2012 09:18:57 PM Peter Geoghegan wrote:
>> On 15 October 2012 19:19, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> > I think Robert is right that if Slony can't use the API, it is unlikely
>> > any other replication system could use it.
>>
>> I don't accept that. Clearly there is a circular dependency, and
>> someone has to go first - why should the Slony guys invest in adopting
>> this technology if it is going to necessitate using a forked Postgres
>> with an uncertain future?
>
> Well. I don't think (hope) anybody proposed making something release worthy for
> slony but rather a POC patch that proofs the API is generic enough to be used
> by them. If I (or somebody else familiar with this) work together with somebody
> familiar with with slony internals I think such a POC shouldn't be too hard to
> do.
> I think some more input from that side is a good idea. I plan to send out an
> email to possibly interested parties in about two weeks...

What Slony essentially sends to the receiver node is a COPY stream in
the format, Christopher described. That stream is directly copied into
the receiving node's sl_log_N table and picked up there by an apply
trigger BEFORE INSERT, that performs the corresponding
INSERT/UPDATE/DELETE operation via prepared plans to the user tables.

For a POC I think it is sufficient to demonstrate that this copy stream
can be generated out of the WAL decoding.

Note that Slony today does not touch columns in an UPDATE, that have not
changed in the original UPDATE on the origin. Sending toasted column
values, that haven't changed, would be a substantial change to the
storage efficiency on the replica. The consequence of this is that the
number of colums that need to be in the UPDATE's SET clause varies. The
log_cmdupdncols is to separate the new column/value pairs from the
column/key pairs of the updated row. The old row "key" in Slony is based
on a unique index (preferably a PK, but any unique key will do). This
makes that cmdupdncols simply the number of column/value pairs minus the
number of key columns. So it isn't too hard to figure out.

Jan

--
Anyone who trades liberty for security deserves neither
liberty nor security. -- Benjamin Franklin