Re: Transactions involving multiple postgres foreign servers

From: Daniel Gustafsson <daniel(at)yesql(dot)se>
To: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>, Vinayak Pokale <pokale_vinayak_q3(at)lab(dot)ntt(dot)co(dot)jp>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Transactions involving multiple postgres foreign servers
Date: 2017-10-02 10:13:18
Message-ID: 6C87E328-116F-4AB0-87C5-803CF933F6E1@yesql.se
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

> On 02 Oct 2017, at 08:31, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Sat, Sep 30, 2017 at 12:42 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> On Wed, Sep 27, 2017 at 11:15 PM, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>>> I think that making a resolver process have connection caches to each
>>> foreign server for a while can reduce the overhead of connection to
>>> foreign servers. These connections will be invalidated by DDLs. Also,
>>> most of the time we spend to commit a distributed transaction is the
>>> interaction between the coordinator and foreign servers using
>>> two-phase commit protocal. So I guess the time in signalling to a
>>> resolver process would not be a big overhead.
>>
>> I agree. Also, in the future, we might try to allow connections to be
>> shared across backends. I did some research on this a number of years
>> ago and found that every operating system I investigated had some way
>> of passing a file descriptor from one process to another -- so a
>> shared connection cache might be possible.
>
> It sounds good idea.
>
>> Also, we might port the whole backend to use threads, and then this
>> problem goes way. But I don't have time to write that patch this
>> week. :-)
>>
>> It's possible that we might find that neither of the above approaches
>> are practical and that the performance benefits of resolving the
>> transaction from the original connection are large enough that we want
>> to try to make it work anyhow. However, I think we can postpone that
>> work to a future time. Any general solution to this problem at least
>> needs to be ABLE to resolve transactions at a later time from a
>> different session, so let's get that working first, and then see what
>> else we want to do.
>
> I understood and agreed. I'll post the first version patch of new
> design to next CF.

Closing this patch with Returned with feedback in this commitfest, looking
forward to a new version in an upcoming commitfest.

cheers ./daniel

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Daniel Gustafsson 2017-10-02 10:18:51 Re: GSoC 2017: weekly progress reports (week 8)
Previous Message Daniel Gustafsson 2017-10-02 10:06:28 Re: Hooks to track changed pages for backup purposes