Re: Transactions involving multiple postgres foreign servers

On Tue, Sep 27, 2016 at 2:54 PM, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> On Mon, Sep 26, 2016 at 9:07 PM, Ashutosh Bapat
> <ashutosh(dot)bapat(at)enterprisedb(dot)com> wrote:
>> On Mon, Sep 26, 2016 at 5:25 PM, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>>> On Mon, Sep 26, 2016 at 7:28 PM, Ashutosh Bapat
>>> <ashutosh(dot)bapat(at)enterprisedb(dot)com> wrote:
>>>> My original patch added code to manage the files for 2 phase
>>>> transactions opened by the local server on the remote servers. This
>>>> code was mostly inspired from the code in twophase.c which manages the
>>>> file for prepared transactions. The logic to manage 2PC files has
>>>> changed since [1] and has been optimized. One of the things I wanted
>>>> to do is see, if those optimizations are applicable here as well. Have
>>>> you considered that?
>>>>
>>>>
>>>
>>> Yeah, we're considering it.
>>> After these changes are committed, we will post the patch incorporated
>>> these changes.
>>>
>>> But what we need to do first is the discussion in order to get consensus.
>>> Since current design of this patch is to transparently execute DCL of
>>> 2PC on foreign server, this code changes lot of code and is
>>> complicated.
>>
>> Can you please elaborate. I am not able to understand what DCL is
>> involved here. According to [1], examples of DCL are GRANT and REVOKE
>> command.
>
> I meant transaction management command such as PREPARE TRANSACTION and
> COMMIT/ABORT PREPARED command.
> The web page I refered might be wrong, sorry.
>
>>> Another approach I have is to push down DCL to only foreign servers
>>> that support 2PC protocol, which is similar to DML push down.
>>> This approach would be more simpler than current idea and is easy to
>>> use by distributed transaction manager.
>>
>> Again, can you please elaborate, how that would be different from the
>> current approach and how does it simplify the code.
>>
>
> The idea is just to push down PREPARE TRANSACTION, COMMIT/ROLLBACK
> PREPARED to foreign servers that support 2PC.
> With this idea, the client need to do following operation when foreign
> server is involved with transaction.
>
> BEGIN;
> UPDATE parent_table SET ...; -- update including foreign server
> PREPARE TRANSACTION 'xact_id';
> COMMIT PREPARED 'xact_id';
>
> The above PREPARE TRANSACTION and COMMIT PREPARED command are pushed
> down to foreign server.
> That is, the client needs to execute PREPARE TRANSACTION and
>
> In this idea, I think that we don't need to do followings,
>
> * Providing the prepare id of 2PC.
> Current patch adds new API prepare_id_provider() but we can use the
> prepare id of 2PC that is used on parent server.
>
> * Keeping track of status of foreign servers.
> Current patch keeps track of status of foreign servers involved with
> transaction but this idea is just to push down transaction management
> command to foreign server.
> So I think that we no longer need to do that.

> COMMIT/ROLLBACK PREPARED explicitly.

The problem with this approach is same as one previously stated. If
the connection between local and foreign server is lost between
PREPARE and COMMIT the prepared transaction on the foreign server
remains dangling, none other than the local server knows what to do
with it and the local server has lost track of the prepared
transaction on the foreign server. So, just pushing down those
commands doesn't work.

>
> * Adding max_prepared_foreign_transactions parameter.
> It means that the number of transaction involving foreign server is
> the same as max_prepared_transactions.
>

That isn't true exactly. max_prepared_foreign_transactions indicates
how many transactions can be prepared on the foreign server, which in
the method you propose should have a cap of max_prepared_transactions
* number of foreign servers.

--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company