| From: | Petr Jelinek <petr(at)2ndquadrant(dot)com> | 
|---|---|
| To: | PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> | 
| Cc: | Craig Ringer <craig(at)2ndquadrant(dot)com> | 
| Subject: | pglogical - logical replication contrib module | 
| Date: | 2015-12-31 23:34:30 | 
| Message-ID: | 5685BB86.5010901@2ndquadrant.com | 
| Views: | Whole Thread | Raw Message | Download mbox | Resend email | 
| Thread: | |
| Lists: | pgsql-hackers | 
Hi,
I'd like to submit the replication solution which is based on the 
pglogical_output [1] module (which is obviously needed for this to compile).
The pglogical contrib module provides extension which does the 
master-slave logical replication based on the logical decoding.
The basic documentation is in README.md, I didn't bother making sgml 
docs yet since I expect that there will be ongoing changes happening and 
it's easier for me to update the markdown docs than sgml. I will do the 
conversion once we start approaching committable state.
What it implements
- logical replication
- partial replication (replication sets)
- multiple sources for single subscriber
- origin filtering (so that if replication is setup both ways, there is 
no cyclic replication)
It currently doesn't do multi-master or automatic DDL. I think DDL 
should be relatively easy if somebody finishes the deparse extension as 
the infrastructure for replicating arbitrary commands is present in this 
patch.
It's rather large patch so I will just go very briefly over high level 
overview of how it works, the details need to be discussed separately IMHO:
Catalogs:
  - node - stores information about "nodes" (postgresql databases)
  - node_interface - represents connection string(s) to nodes, we 
separate interfaces to different catalog mainly to allow for setups 
where different subscribers see different address of the provider server
  - local_node - stores exactly one tuple which points to the nodes 
catalog tuple that represents the local node of the current database
  - subscription - represents subscription between two nodes, it 
includes configuration of the subscription like replication set and 
origin filters
Upstream:
  - basically just implements the pglogical_output hooks according to 
the catalogs
Downstream:
  - several background workers
  - supervisor is worker which manages all the other workers
  - manager is per database worker which manages individual database 
(node) and its subscriptions
  - apply does the actual replication, one apply process is started per 
subscription, connects to the walsender on the other side and applies 
the changes received from there
[1] https://commitfest.postgresql.org/8/418/
-- 
  Petr Jelinek                  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services
| Attachment | Content-Type | Size | 
|---|---|---|
| 0001-pglogical-v1.patch.gz | application/gzip | 93.9 KB | 
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Petr Jelinek | 2015-12-31 23:51:51 | Re: Sequence Access Method WIP | 
| Previous Message | Peter Geoghegan | 2015-12-31 23:08:34 | Re: PATCH: Extending the HyperLogLog API a bit |