Re: Logical replication and multimaster

On 3 December 2015 at 20:39, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:

> On 30 November 2015 at 17:20, Konstantin Knizhnik <
> k(dot)knizhnik(at)postgrespro(dot)ru> wrote:
>
>
>> But looks like there is not so much sense in having multiple network
>> connection between one pair of nodes.
>> It seems to be better to have one connection between nodes, but provide
>> parallel execution of received transactions at destination side. But it
>> seems to be also nontrivial. We have now in PostgreSQL some infrastructure
>> for background works, but there is still no abstraction of workers pool and
>> job queue which can provide simple way to organize parallel execution of
>> some jobs. I wonder if somebody is working now on it or we should try to
>> propose our solution?
>>
>
> There are definitely two clear places where additional help would be
> useful and welcome right now.
>
> 1. Allowing logical decoding to have a "speculative pre-commit data"
> option, to allow some data to be made available via the decoding api,
> allowing data to be transferred prior to commit.
>

Something relevant I ran into re this:

in reorderbuffer.c, on ReorderBufferCommit:

* We currently can only decode a transaction's contents in when their
commit
* record is read because that's currently the only place where we know
about
* cache invalidations. Thus, once a toplevel commit is read, we iterate
over
* the top and subtransactions (using a k-way merge) and replay the
changes in
* lsn order.

I haven't dug into the implications particularly as I'm chasing something
else, but want to note it on the thread. Here be dragons when it comes to
transaction streaming.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services