| From: | Masahiro Ikeda <ikedamsh(at)oss(dot)nttdata(dot)com> |
|---|---|
| To: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com> |
| Cc: | Zhihong Yu <zyu(at)yugabyte(dot)com>, Ibrar Ahmed <ibrar(dot)ahmad(at)gmail(dot)com>, Masahiko Sawada <masahiko(dot)sawada(at)2ndquadrant(dot)com>, "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, "ashutosh(dot)bapat(dot)oss(at)gmail(dot)com" <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, "amit(dot)kapila16(at)gmail(dot)com" <amit(dot)kapila16(at)gmail(dot)com>, "m(dot)usama(at)gmail(dot)com" <m(dot)usama(at)gmail(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, "sulamul(at)gmail(dot)com" <sulamul(at)gmail(dot)com>, "alvherre(at)2ndquadrant(dot)com" <alvherre(at)2ndquadrant(dot)com>, "thomas(dot)munro(at)gmail(dot)com" <thomas(dot)munro(at)gmail(dot)com>, "ildar(at)adjust(dot)com" <ildar(at)adjust(dot)com>, "horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp" <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>, "chris(dot)travers(at)adjust(dot)com" <chris(dot)travers(at)adjust(dot)com>, "robertmhaas(at)gmail(dot)com" <robertmhaas(at)gmail(dot)com>, "ishii(at)sraoss(dot)co(dot)jp" <ishii(at)sraoss(dot)co(dot)jp> |
| Subject: | Re: Transactions involving multiple postgres foreign servers, take 2 |
| Date: | 2021-06-25 01:33:23 |
| Message-ID: | 1d7b5f65-0dd3-fe3c-506a-3fcd51b07e95@oss.nttdata.com |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On 2021/06/24 22:11, Masahiko Sawada wrote:
> On Sat, Jun 12, 2021 at 1:25 AM Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com> wrote:
>> On 2021/05/11 13:37, Masahiko Sawada wrote:
>> So currently FdwXactInsertEntry() calls XLogInsert() and XLogFlush() for
>> XLOG_FDWXACT_INSERT WAL record. Additionally we should also wait there
>> for WAL record to be replicated to the standby if sync replication is enabled?
>> Otherwise, when the failover happens, new primary (past-standby)
>> might not have enough XLOG_FDWXACT_INSERT WAL records and
>> might fail to find some in-doubt foreign transactions.
>
> But even if we wait for the record to be replicated, this problem
> isn't completely resolved, right? If the server crashes before the
> standy receives the record and the failover happens then the new
> master doesn't have the record. I wonder if we need to have another
> FDW API in order to get the list of prepared transactions from the
> foreign server (FDW). For example in postgres_fdw case, it gets the
> list of prepared transactions on the foreign server by executing a
> query. It seems to me that this corresponds to xa_recover in the XA
> specification.
FWIF, Citus implemented as sawada-san said above [1].
Since each WAL record for PREPARE is flushing in the latest patch, the latency
became too much, especially under synchronous replication. For example, the
transaction involving three foreign servers must wait to sync "three" WAL
records for PREPARE and "one" WAL records for local commit in remote server
one by one sequentially. So, I think that Sawada-san's idea is good to improve
the latency although fdw developer's work increases.
[1]
SIGMOD 2021 525 Citus: Distributed PostgreSQL for Data Intensive Applications
From 12:27 says that how to solve unresolved prepared xacts.
https://www.youtube.com/watch?v=AlF4C60FdlQ&list=PL3xUNnH4TdbsfndCMn02BqAAgGB0z7cwq
Regards,
--
Masahiro Ikeda
NTT DATA CORPORATION
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Peter Geoghegan | 2021-06-25 01:33:59 | Re: Using indexUnchanged with nbtree |
| Previous Message | David Rowley | 2021-06-25 01:26:08 | Re: Deadlock risk while inserting directly into partition? |