Re: logical changeset generation v4

From: Steve Singer <steve(at)ssinger(dot)info>
To: Andres Freund <andres(at)2ndquadrant(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: logical changeset generation v4
Date: 2013-01-20 04:42:02
Message-ID: BLU0-SMTP7022C3F56453DB72B4034EDC100@phx.gbl
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 13-01-14 08:38 PM, Andres Freund wrote:
> Hi everyone,
> Here is the newest version of logical changeset generation.
> 2) Currently the logical replication infrastructure assigns a 'slot-id'
> when a new replica is setup. That slot id isn't really nice
> (e.g. "id-321578-3"). It also requires that [18] keeps state in a global
> variable to make writing regression tests easy.
> I think it would be better to make the user specify those replication
> slot ids, but I am not really sure about it.

Shortly after trying out the latest version I hit the following scenario
1. I started pg_receivellog but mistyped the name of my plugin
2. It looped and used up all of my logical replication slots

I killed pg_receivellog and restarted it with the correct plugin name
but it won't do anything because I have no free slots. I can't free the
slots with -F because I have no clue what the names of the slots are.
I can figure the names out by looking in pg_llog but if my replication
program can't do that so it won't be able to clean up from a failed attempt.

I agree with you that we should make the user program specify a slot, we
eventually might want to provide a view that shows the currently
allocated slots. For a logical based slony I would just generate the
slot name based on the remote node id. If walsender generates the slot
name then I would need to store a mapping between slot names and slons
so when a slon restarted it would know which slot to resume using. I'd
have to use a table in the slony schema on the remote database for
this. There would always be a risk of losing track of a slot id if the
slon crashed after getting the slot number but before committing the
mapping on the remote database.

> 3) Currently no options can be passed to an output plugin. I am thinking
> about making "INIT_LOGICAL_REPLICATION 'plugin'" accept the now widely
> used ('option' ['value'], ...) syntax and pass that to the output
> plugin's initialization function.

I think we discussed this last CF, I like this idea.

> 4) Does anybody object to:
> -- allocate a permanent replication slot
> INIT_LOGICAL_REPLICATION 'plugin' 'slotname' (options);
> -- stream data
> START_LOGICAL_REPLICATION 'slotname' 'recptr';
> -- deallocate a permanent replication slot


> 5) Currently its only allowed to access catalog tables, its fairly
> trivial to extend this to additional tables if you can accept some
> (noticeable but not too big) overhead for modifications on those tables.
> I was thinking of making that an option for tables, that would be useful
> for replication solutions configuration tables.

I think this will make the life of anyone developing a new replication
system easier. Slony has a lot of infrastructure for allowing slonik
scripts to wait for configuration changes to popogate everywhere before
making other configuration changes because you can get race conditions.
If I were designing a new replication system and I had this feature then
I would try to use it to come up with a simpler model of propagating
configuration changes.

> Andres Freund


In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Xi Wang 2013-01-20 04:51:26 [PATCH] Fix NULL checking in check_TSCurrentConfig()
Previous Message wln 2013-01-20 03:37:54