Re: Design for In-Core Logical Replication

On 22 July 2016 at 22:55, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:

>
> Filtering data at both the sender and receiver has been requested, so
> I guess it makes sense to have a WHERE clause on both the Publication
> and the Subscription, but I will think more on that.
>

Yeah, output plugin level filtering predicates are certainly a desirable
feature for down the track.

One important factor to consider there is that we're very restricted in
what we can safely access. Currently the historical snapshot infrastructure
set up by logical decoding doesn't stop you trying to access tables that
you can't safely access, you might just get surprising/wrong results or
failures. If we're going to allow arbitrary user defined predicates we'll
likely need to change that so we explicitly ERROR when an attempt to
heap_open a table that isn't accessible during logical decoding is made.
Probably not hard, likely a good idea anyway.

Such predicates should generally be simple, though with less strict
requirements than CHECK constraints. We can probably do subqueries and I
don't think functions have to be immutable, though obviously anything that
tries to allocate an xid will fail.

Transforming data at the receiver/Apply side we had envisaged would be
> done using standard triggers. So we had envisaged you would add the
> trigger onto the table using the existing trigger syntax, so that when
> an INSERT was applied the trigger would execute, using the standard
> mechanisms.
>

Yeah. In the medium term at least.

I imagine eventually we'll want more powerful transforms that doesn't
require the overhead of trigger function calls, but that's a looooong way
down the road.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services