Re: Replication identifiers, take 4

On 02/16/2015 11:18 AM, Andres Freund wrote:
> On 2015-02-16 11:07:09 +0200, Heikki Linnakangas wrote:
>> How does this work if you have multiple replication systems in use in the
>> same cluster? You might use Slony to replication one table to one system,
>> and BDR to replication another table with another system. Or the same
>> replication software, but different hosts.
>
> It should "just work". Replication identifiers are identified by a free
> form text, each replication solution can add the
> information/distinguising data they need in there.
>
> Bdr uses something like
> #define BDR_NODE_ID_FORMAT "bdr_"UINT64_FORMAT"_%u_%u_%u_%s"
> with
> remote_sysid, remote_tlid, remote_dboid, MyDatabaseId, configurable_name
> as parameters as a replication identifier name.
>
> I've been wondering whether the bdr_ part in the above should be split
> of into a separate field, similar to how the security label stuff does
> it. But I don't think it'd really buy us much, especially as we did
> not do that for logical slot names.
>
> Each of the used replication solution would probably ask their output
> plugin to only stream locally generated (i.e. origin_id =
> InvalidRepNodeId) changes, and possibly from a defined list of other
> known hosts in the cascading case.
>
> Does that answer your question?

Yes, thanks. Note to self and everyone else looking at this: It's
important to keep in mind is that the replication IDs are completely
internal to the local cluster. They are *not* sent over the wire.

That's not completely true if you also use physical replication, though.
The replication IDs will be included in the WAL stream. Can you have
logical decoding running in a hot standby server? And how does the
progress report stuff that's checkpointed in pg_logical/checkpoints work
in a hot standby? (Sorry if I'm not making sense, I haven't really
thought hard about this myself)

At a quick glance, this basic design seems workable. I would suggest
expanding the replication IDs to regular 4 byte oids. Two extra bytes is
a small price to pay, to make it work more like everything else in the
system.

- Heikki