Re: repeated decoding of prepared transactions

Hi,

On 2021-02-19 15:53:32 +0100, Markus Wanner wrote:
> However, more generally speaking, I suspect you are overthinking this. All
> of the complexity arises because of the assumption that an output plugin
> receiving and confirming a PREPARE may not be able to persist that first
> phase of transaction application. Instead, you are trying to somehow
> resurrect the transactional changes and the prepare at COMMIT PREPARED time
> and decode it in a deferred way.

The output plugin should never persist anything. That's the job of the
client, not the output plugin. The output plugin simply doesn't have the
information to know whether the client received data and successfully
applied data or not.

> Given the output plugin supports two-phase commit, I argue there must be a
> good reason for it setting the start_decoding_at LSN to a point in time
> after a PREPARE. To me that means the output plugin (or its downstream
> replica) has processed the PREPARE (and the downstream replica did whatever
> it needed to do on its side in order to make the transaction ready to be
> committed in a second phase).

The output plugin doesn't set / influence start_decoding_at (unless you
want to count just ERRORing out).

> With that line of thinking, the point in time (or in WAL) of the COMMIT
> PREPARED does not matter at all to reason about the decoding of the PREPARE
> operation. Instead, there are only exactly two cases to consider:
>
> a) the PREPARE happened before the start_decoding_at LSN and must not be
> decoded. (But the effects of the PREPARE must then be included in the
> initial synchronization. If that's not supported, the output plugin should
> not enable two-phase commit.)

I don't think that can be made work without disproportionate
complexity. Especially not in cases where we start to be CONSISTENT
based on pre-existing on-disk snapshots.

Greetings,

Andres Freund