repeated decoding of prepared transactions

From: Markus Wanner <markus(dot)wanner(at)enterprisedb(dot)com>
To: pgsql-hackers(at)postgresql(dot)org, amit(dot)kapila16(at)gmail(dot)com, itsajin(at)gmail(dot)com
Subject: repeated decoding of prepared transactions
Date: 2021-02-08 08:31:12
Message-ID: d0f60d60-133d-bf8d-bd70-47784d8fabf3@enterprisedb.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Amit, Ajin, hackers,

testing logical decoding for two-phase transactions, I stumbled over
what I first thought is a bug. But comments seems to indicate this is
intended behavior. Could you please clarify or elaborate on the design
decision? Or indicate this indeed is a bug?

What puzzled me is that if a decoder is restarted in between the PREPARE
and the COMMIT PREPARED, it repeats the entire transaction, despite it
being already sent and potentially prepared on the receiving side.

In terms of `pg_logical_slot_get_changes` (and roughly from the
prepare.sql test), this looks as follows:

data
----------------------------------------------------
BEGIN
table public.test_prepared1: INSERT: id[integer]:1
PREPARE TRANSACTION 'test_prepared#1'
(3 rows)

This is the first delivery of the transaction. After a restart, it will
get all of the changes again, though:

data
----------------------------------------------------
BEGIN
table public.test_prepared1: INSERT: id[integer]:1
PREPARE TRANSACTION 'test_prepared#1'
COMMIT PREPARED 'test_prepared#1'
(4 rows)

I did not expect this, as any receiver that wants to have decoded 2PC is
likely supporting some kind of two-phase commits itself. And would
therefore prepare the transaction upon its first reception. Potentially
receiving it a second time would require complicated filtering on every
prepared transaction.

Furthermore, this clearly and unnecessarily holds back the restart LSN.
Meaning even just a single prepared transaction can block advancing the
restart LSN. In most cases, these are short lived. But on the other
hand, there may be an arbitrary amount of other transactions in between
a PREPARE and the corresponding COMMIT PREPARED in the WAL. Not being
able to advance over a prepared transaction seems like a bad thing in
such a case.

I fail to see where this repetition would ever be useful. Is there any
reason for the current implementation that I'm missing or can this be
corrected? Thanks for elaborating.

Regards

Markus

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro Horiguchi 2021-02-08 08:32:46 Re: Is Recovery actually paused?
Previous Message Tang, Haiying 2021-02-08 08:12:40 RE: Parallel INSERT (INTO ... SELECT ...)