Re: Logical replication keepalive flood

On Thu, Sep 30, 2021 at 6:08 PM Kyotaro Horiguchi
<horikyota(dot)ntt(at)gmail(dot)com> wrote:
>
> At Thu, 30 Sep 2021 17:21:03 +1000, Greg Nancarrow <gregn4422(at)gmail(dot)com> wrote in
> > Actually, with the patch applied, I find that "make check-world" fails
> > (006_logical_decoding, test 7).
>
> Mmm..
>
> t/006_logical_decoding.pl .. 4/14
> # Failed test 'pg_recvlogical acknowledged changes'
> # at t/006_logical_decoding.pl line 117.
> # got: 'BEGIN
> # table public.decoding_test: INSERT: x[integer]:5 y[text]:'5''
> # expected: ''
>
> I'm not sure what the test is checking for now, though.
>

I think it's trying to check that pg_recvlogical doesn't read "past" a
specified "--endpos" LSN. The test is invoking pg_recvlogical with the
same --endpos LSN value multiple times.
After first getting the LSN (to use for the --endpos value) after 4
rows are inserted, some additional rows are inserted which the test
expects pg_recvlogical won't read because it should't read past
--endpos.
Problem is, the test seems to be relying on a keepalive between the
WAL record of the first transaction and the WAL record of the next
transaction.
As Amit previously explained on this thread "I think here the reason
is that the first_lsn of a transaction is always equal to end_lsn of
the previous transaction (See comments
above first_lsn and end_lsn fields of ReorderBufferTXN)."
When the patch is applied, pg_recvlogical doesn't read a keepalive
when it is invoked with the same --endpos for a second time here, and
it ends up reading the first WAL record for the next transaction
(those additional rows that the test expects it won't read).

Regards,
Greg Nancarrow
Fujitsu Australia