Re: Minimal logical decoding on standbys

Hi,

On 1/24/23 3:31 PM, Drouvot, Bertrand wrote:
> Hi,
>
> On 1/24/23 6:20 AM, Drouvot, Bertrand wrote:
>> Hi,
>>
>> On 1/24/23 1:46 AM, Andres Freund wrote:
>>> Hi,
>>>
>>> On 2023-01-19 10:43:27 +0100, Drouvot, Bertrand wrote:
>>>> Sure, so with:
>>>>
>>>> 1) hot_standby_feedback set to off on the standby
>>>> 2) create 2 logical replication slots on the standby and activate one
>>>> 3) Invalidate the logical slots on the standby with VACUUM FULL on the primary
>>>> 4) change hot_standby_feedback to on on the standby
>>>>
>>>> If:
>>>>
>>>> 5) pg_reload_conf() on the standby, then on the primary we get a catalog_xmin
>>>> for the physical slot that the standby is attached to:
>>>>
>>>> postgres=# select slot_type,xmin,catalog_xmin  from pg_replication_slots ;
>>>>   slot_type | xmin | catalog_xmin
>>>> -----------+------+--------------
>>>>   physical  |  822 |          748
>>>> (1 row)
>>>
>>> How long did you wait for this to change?
>>
>> Almost instantaneous after pg_reload_conf() on the standby.
>>
>>> I don't think there's anything right
>>> now that'd force a new hot-standby-feedback message to be sent to the primary,
>>> after slots got invalidated.
>>>
>>> I suspect that if you terminated the walsender connection on the primary,
>>> you'd not see it anymore either?
>>>
>>
>> Still there after the standby is shutdown but disappears when the standby is re-started.
>>
>>> If that isn't it, something is broken in InvalidateObsolete...
>>>
>
> Yeah, you are right: ReplicationSlotsComputeRequiredXmin() is missing for the
> logical slot invalidation case (and ReplicationSlotsComputeRequiredXmin() also
> needs to take care of it).
>
> I'll provide a fix in the next revision along with the TAP tests comments addressed.
>

Please find attached V43 addressing the comments related to the TAP tests (in 0004 at that time) that have been done in [1] .

Remarks:

- The C helper function to call LogStandbySnapshot() is not done yet.
- While working on it, I discovered that the new isCatalogRel field was not populated in gistXLogDelete(): fixed in V43.
- The issue described above is also fixed so that a standby restart or a reload would produce the same behavior
on the primary physical slot (aka catalog_xmin is empty if logical slots are invalidated).
- A test with pg_recvlogical started before the standby promotion has been added.
- A test for conflict due to row removal via on-access pruning has been added.
- I'm struggling to create a test for btree killtuples as there is a need for rows removal on the table (that could produce a conflict too):
Do you've a scenario in mind for this one? (and btw in what kind of WAL record should the conflict be detected in such a case? xl_btree_delete?)

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

[1]: https://www.postgresql.org/message-id/20230106034036.2m4qnn7ep7b5ipet%40awork3.anarazel.de