Re: Logical replication existing data copy

From: Petr Jelinek <petr(dot)jelinek(at)2ndquadrant(dot)com>
To: Erik Rijkers <er(at)xs4all(dot)nl>
Cc: Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, pgsql-hackers-owner(at)postgresql(dot)org
Subject: Re: Logical replication existing data copy
Date: 2017-03-06 10:27:43
Message-ID: 837653af-ec47-dacb-ae33-4a52441fd9ab@2ndquadrant.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

updated and rebased version of the patch attached.

I changed a bit how the table handover is done between tablesync worker
and apply worker.

I also changed tablesync worker to do whole thing including catchup in
single transaction, that might mean more bloat on very active servers
but it's currently the only way to make sure that if it fails it can
restart as otherwise there would be leftover data from previous attempt
with no way of continuing from the point of failure. This means that
some of the synchronization states now only exist in memory and are
never saved in catalog, I separated those to make that clear. Hopefully
this didn't break anything.

I also fixed thinko about how we handle the LSN returned by
CREATE_REPLICATION_SLOT which should fix the missing row issue Erik
reported earlier (this time it wasn't snapbuild.c issue :) ).

And lastly I changed the automagic around exporting, not exporting and
using the snapshot produced by CREATE_REPLICATION_SLOT into explicit
parameter for the CREATE_REPLICATION_SLOT command (and added simple
framework for adding more of those if needed in the future).

--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Attachment Content-Type Size
0001-Logical-replication-support-for-initial-data-copy-v6.patch text/plain 161.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amos Bird 2017-03-06 11:24:26 PATCH: psql show index with type info
Previous Message Petr Jelinek 2017-03-06 10:13:48 Re: Logical Replication and Character encoding