Re: Minimal logical decoding on standbys

From: "Drouvot, Bertrand" <bdrouvot(at)amazon(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Ibrar Ahmed <ibrar(dot)ahmad(at)gmail(dot)com>, Amit Khandekar <amitdkhan(dot)pg(at)gmail(dot)com>, <fabriziomello(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, tushar <tushar(dot)ahuja(at)enterprisedb(dot)com>, "[pgdg] Robert Haas" <robertmhaas(at)gmail(dot)com>, Rahila Syed <rahila(dot)syed(at)2ndquadrant(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Minimal logical decoding on standbys
Date: 2021-09-08 10:08:43
Message-ID: 09dd62cd-6f07-e15b-164c-4ffb1569f837@amazon.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Andres,

On 8/6/21 1:27 PM, Drouvot, Bertrand wrote:
> Hi,
>
> On 8/2/21 6:01 PM, Andres Freund wrote:
>> While working on this I found a, somewhat substantial, issue:
>>> If so, do you already have in mind a way to handle this? (I thought you
>>> already had in mind a way to handle it so the question)
>> Yes. I think we need to add a condition variable to be able to wait for
>> WAL positions to change. Either multiple condition variables (one for
>> the flush position, one for the replay position), or one that just
>> changes more often. That way one can wait for apply without a race
>> condition.
>>
> Thanks for the feedback.
>
> Wouldn't a condition variable on the replay position be enough? I
> don't get why the proposed one on the flush position is needed.

Please find enclosed a patch proposal to address those corner cases.

I think (but may be wrong) that the condition variable on the flush
position would be needed only for the walsender(s) on non Standby node,
that's why:

* I made use of a condition variable on the replay position only.

* The walsender waits on it in WalSndWaitForWal() only if recovery is
in progress.

For simplicity to discuss those corner cases, this is a dedicated patch
that can be applied on top of v23 patches shared previously.

Thanks

Bertrand

>
>>> if not, what kind of additional 
tests would you like to see?
>> A few catalog rows being removed (e.g. due to DELETE and then VACUUM
>> *without* full) and a standby without hot_standby_feedback catching
>> that.
>
> Test added in v23 attached.
>
> Thanks
>
> Bertrand
>

Attachment Content-Type Size
v23-0006-Logical-Decoding-On-Standby-WalSender-Corner-Case.patch text/plain 6.0 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message wangsh.fnst@fujitsu.com 2021-09-08 10:16:46 drop tablespace failed when location contains .. on win32
Previous Message tanghy.fnst@fujitsu.com 2021-09-08 10:07:38 RE: [PATCH] Add tab-complete for backslash commands