Re: [HACKERS] logical decoding of two-phase transactions

On Thu, Feb 18, 2021 at 5:48 AM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
>
> Please find attached the new patch set v41*
>

I see one issue here. Currently, when we create a subscription, we
first launch apply-worker and create the main apply worker slot and
then launch table sync workers as required. Now, assume, the apply
worker slot is created and after that, we launch tablesync worker,
which will initiate its slot (sync_slot) creation. Then, on the
publisher-side, the situation is such that there is a prepared
transaction that happens before we reach a consistent snapshot. We can
assume the exact scenario as we have in twophase_snapshot.spec where
we skip prepared xact due to this reason.

Because the WALSender corresponding to apply worker is already running
so it will be in consistent state, for it, such a prepared xact can be
decoded and it will send the same to the subscriber. On the
subscriber-side, it can skip applying the data-modification operations
because the corresponding rel is still not in a ready state (see
should_apply_changes_for_rel and its callers) simply because the
corresponding table sync worker is not finished yet. But prepare will
occur and it will lead to a prepared transaction on the subscriber.

In this situation, tablesync worker has skipped prepare because the
snapshot was not consistent and then it exited because it is in sync
with the apply worker. And apply worker has skipped because tablesync
was in-progress. Later when Commit prepared will come, the
apply-worker will simply commit the previously prepared transaction
and we will never see the prepared transaction data.

So, the basic premise is that we can't allow tablesync workers to skip
prepared transactions (which can be processed by apply worker) and
process later commits.

I have one idea to address this. When we get the first begin_prepare
in the apply-worker, we can check if there are any relations in
"not_ready" state and if so then just wait till all the relations
become in sync with the apply worker. This is to avoid that any of the
tablesync workers might skip prepared xact and we don't want apply
worker to also skip the same.

Now, it is possible that some tablesync worker has copied the data and
moved the sync position ahead of where the current apply worker's
position is. In such a case, we need to process transactions in apply
worker such that we can process commits if any, and write prepared
transactions to file. For prepared transactions, we can take decisions
only once the commit prepared for them has arrived.

--
With Regards,
Amit Kapila.