Re: logical replication and PANIC during shutdown checkpoint in publisher

From: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
To: Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>
Cc: Petr Jelinek <petr(dot)jelinek(at)2ndquadrant(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: logical replication and PANIC during shutdown checkpoint in publisher
Date: 2017-05-05 05:26:00
Message-ID: CAB7nPqQ-miZy7R2+uMJRZ26spS14E5ioHv7HFZQN_f5w0Ef6KQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, May 3, 2017 at 12:25 AM, Peter Eisentraut
<peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:
> On 5/2/17 10:08, Michael Paquier wrote:
>> On Tue, May 2, 2017 at 9:30 PM, Peter Eisentraut
>> <peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:
>>> On 5/2/17 03:11, Petr Jelinek wrote:
>>>> logical decoding can theoretically
>>>> do HOT pruning (even if the chance is really small) so it's not safe to
>>>> start logical replication either.
>>>
>>> This seems a bit impossible to resolve. On the one hand, we want to
>>> allow streaming until after the shutdown checkpoint. On the other hand,
>>> streaming itself might produce new WAL.
>>
>> It would be nice to split things into two:
>> - patch 1 adding the signal handling that wins a backpatch.
>> - patch 2 fixing the side cases with logical decoding.
>
> The side cases with logical decoding are also not new and would need
> backpatching, AIUI.

Okay, I thought that there was some new concept part of logical
replication here.

>>> Can we prevent HOT pruning during logical decoding?
>>
>> It does not sound much difficult to do, couldn't you just make it a
>> no-op with am_walsender?
>
> That's my hope.

The only code path doing HOT-pruning and generating WAL is
heap_page_prune(). Do you think that we need to worry about FPWs as
well?

Attached is an updated patch, which also forbids the run of any
replication commands when the stopping state is reached.
--
Michael

Attachment Content-Type Size
v4-0001-Prevent-panic-during-shutdown-checkpoint.patch text/x-patch 14.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2017-05-05 05:34:14 Re: Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)
Previous Message Amit Kapila 2017-05-05 05:12:08 Re: statement_timeout is not working as expected with postgres_fdw