Subscription code improvements

From: Petr Jelinek <petr(dot)jelinek(at)2ndquadrant(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
Subject: Subscription code improvements
Date: 2017-07-07 20:19:19
Message-ID: 4c6e7ab3-091e-6336-df38-28937b153e64@2ndquadrant.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I have done some review of subscription handling (well self-review) and
here is the result of that (It's slightly improved version from another
thread [1]).

I split it into several patches:

0001 - Makes subscription worker exit nicely when there is subscription
missing (ie was removed while it was starting) and also makes disabled
message look same as the message when disabled state was detected while
worker is running. This is mostly just nicer behavior for user (ie no
unexpected errors in log when you just disabled the subscription).

0002 - This is bugfix - the sync worker should exit when waiting for
apply and apply dies otherwise there is possibility of not being
correctly synchronized.

0003 - Splits the schizophrenic SetSubscriptionRelState function into
two which don't try to do broken upsert and report proper errors instead.

0004 - Solves the issue which Masahiko Sawada reported [2] about ALTER
SUBSCRIPTION handling of workers not being transactional - we now do
killing of workers transactionally (and we do the same when possible in
DROP SUBSCRIPTION).

0005 - Bugfix to allow syscache access to subscription without being
connected to database - this should work since subscription is pinned
catalog, it wasn't caught because nothing in core is using it (yet).

0006 - Makes the locking of subscriptions more granular (this depends on
0005). This removes the ugly AccessExclusiveLock on the pg_subscription
catalog from DROP SUBSCRIPTION and also solves some potential race
conditions between launcher, ALTER SUBSCRIPTION and
process_syncing_tables_for_apply(). Also simplifies memory handling in
launcher as well as logicalrep_worker_stop() function. This basically
makes subscriptions behave like every other object in the database in
terms of locking.

Only the 0002, 0004 and 0005 are actual bug fixes, but I'd still like to
get it all into PG10 as especially the locking now behaves really
differently than everything else and that does not seem like a good idea.

[1]
https://www.postgresql.org/message-id/flat/3e06c16c-f441-c606-985c-6d6239097f54(at)2ndquadrant(dot)com
[2]
https://www.postgresql.org/message-id/flat/CAD21AoBD4T2RwTiWD8YfXd+q+m9swX50YjqT5ibj2MzEBVnBhw(at)mail(dot)gmail(dot)com

--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Attachment Content-Type Size
0001-Improve-messaging-during-logical-replication-worker-.patch text/x-patch 2.0 KB
0002-Exit-in-sync-worker-if-relation-was-removed-during-s.patch text/x-patch 1.4 KB
0003-Split-the-SetSubscriptionRelState-function-into-two.patch text/x-patch 10.9 KB
0004-Only-kill-sync-workers-at-commit-time-in-SUBSCRIPTIO.patch text/x-patch 6.5 KB
0005-Allow-syscache-access-to-subscriptions-in-database-l.patch text/x-patch 1.0 KB
0006-Improve-locking-for-subscriptions-and-subscribed-rel.patch text/x-patch 23.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Petr Jelinek 2017-07-07 20:28:21 Re: More race conditions in logical replication
Previous Message Alvaro Herrera 2017-07-07 19:59:56 Re: [WIP] Zipfian distribution in pgbench