Re: Logical Replication WIP

From: Petr Jelinek <petr(at)2ndquadrant(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Steve Singer <steve(at)ssinger(dot)info>, Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>, Craig Ringer <craig(at)2ndquadrant(dot)com>, Simon Riggs <simon(at)2ndquadrant(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Logical Replication WIP
Date: 2016-11-20 18:02:52
Message-ID: 7d6e2e70-f72c-6180-92b7-9908d3d7f8a7@2ndquadrant.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

attached is v8. No tarballing this time ;)

About the patches:

0001:
This is the reworked approach to temporary slots that I sent earlier.

0002:
I ripped out the libpq_select completely and did what Andres suggested,
ie, WaitLatchOrSocket, that needed changes for WalReceiver to use
procLatch but that was trivial. Otherwise it's same.

0003:
Changes:
- Moved the parser of options into C
- Removed the dead references to "FOR TABLE ALL IN SCHEMA"
- Rephrased some things and fixed several typos
- Added needed check into ALTER TABLE ... SET UNLOGGED
- Fixed the UPDATE/DELETE check in CheckCmdReplicaIdentity
- Added owner
- Fixed permission checks

I didn't do any of the text array instead of bools and
objclassid/objsubid as the reasoning for former is wrong IMHO and the
latter is quite premature and I am still not convinced it will be ever
needed.

I also didn't do couple of things reported by PeterE:
> relcache.c: pubactions->pubinsert |= pubform->pubinsert; etc. should be ||=

This one does not seem to be true, there is no ||= and |= works fine for
booleans.

And
> The code for OCLASS_PUBLICATION_REL in
> getObjectIdentityParts() does not fill in objname and objargs, as it is
> supposed to.

From what I see it already does that.

0004:
Changes:
- Added separate DropSubscriptionStmt statement for DROP. This was
prompted by Andres' comment about event triggers. The event triggers
actually work fine as all the SQL is only supposed to touch
subscriptions in current database even though it's shared catalog (it's
only shared because we need the catalog pin but that's implementation
detail), but the DROP would break if name matched subscription in
another database if handled by DropStmt.
- Added SLOT_DROP/NOSLOT_DROP options to DROP SUBSCRIPTION, the new
DropSubscriptionStmt helps here as well
- Added owner
- Moved the option parsing into C

0005/0006 - Mainly just included the doc patch from Steve Singer and did
some additional doc fixes.

The 0007 is something that's more a question for discussion if we want
that. It adds new GUC that sets synchronous commit for apply workers and
defaults to off. This gives quite noticeable performance boost while
still working correctly even is provider uses sync replication. This is
based on experience (and default behaviour) of BDR and pglogical, but I
am not quite sure if core postgres should have that as well (I think it
definitely should have the option, question is more about default setting).

And that's it for now. After some discussion with PeterE I decided to
skip the initial sync patch as it has quite high impact on development
of rest of the patch (because it touches everything, I spend all my time
rebasing it instead of actually fixing things) and can be done as
follow-up patch. I also believe that it will also be polished much
faster once I can fully concentrate on it when this part is done.

--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Attachment Content-Type Size
0001-Add-support-for-TEMPORARY-replication-slots-v8.patch.gz application/gzip 7.6 KB
0002-Refactor-libpqwalreceiver-v8.patch.gz application/gzip 8.8 KB
0003-Add-PUBLICATION-catalogs-and-DDL-v8.patch.gz application/gzip 29.1 KB
0004-Add-SUBSCRIPTION-catalog-and-DDL-v8.patch.gz application/gzip 26.5 KB
0005-Define-logical-replication-protocol-and-output-plugi-v8.patch.gz application/gzip 12.7 KB
0006-Add-logical-replication-workers-v8.patch.gz application/gzip 41.4 KB
0007-Add-separate-synchronous-commit-control-for-logical--v8.patch.gz application/gzip 1.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Petr Jelinek 2016-11-20 18:06:00 Re: Logical Replication WIP
Previous Message Vik Fearing 2016-11-20 17:23:12 Re: Mail thread references in commits