Re: Transactions involving multiple postgres foreign servers, take 2

From: Masahiko Sawada <masahiko(dot)sawada(at)2ndquadrant(dot)com>
To: Masahiro Ikeda <ikedamsh(at)oss(dot)nttdata(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Tatsuo Ishii <ishii(at)sraoss(dot)co(dot)jp>, Muhammad Usama <m(dot)usama(at)gmail(dot)com>, amul sul <sulamul(at)gmail(dot)com>, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, Sawada Masahiko <sawada(dot)mshk(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Ildar Musin <ildar(at)adjust(dot)com>, Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>, Chris Travers <chris(dot)travers(at)adjust(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>
Subject: Re: Transactions involving multiple postgres foreign servers, take 2
Date: 2020-06-23 03:33:02
Message-ID: CA+fd4k58+qZPVf=CwEsC+qGd7XwM=M-jgzLxRSK30Y+UB0cA+Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, 17 Jun 2020 at 14:07, Masahiko Sawada
<masahiko(dot)sawada(at)2ndquadrant(dot)com> wrote:
>
> On Wed, 17 Jun 2020 at 09:01, Masahiro Ikeda <ikedamsh(at)oss(dot)nttdata(dot)com> wrote:
> >
> > > I've attached the new version patch set. 0006 is a separate patch
> > > which introduces 'prefer' mode to foreign_twophase_commit.
> >
> > I hope we can use this feature. Thank you for making patches and
> > discussions.
> > I'm currently understanding the logic and found some minor points to be
> > fixed.
> >
> > I'm sorry if my understanding is wrong.
> >
> > * The v22 patches need rebase as they can't apply to the current master.
> >
> > * FdwXactAtomicCommitParticipants said in
> > src/backend/access/fdwxact/README
> > is not implemented. Is FdwXactParticipants right?
>
> Right.
>
> >
> > * A following comment says that this code is for "One-phase",
> > but second argument of FdwXactParticipantEndTransaction() describes
> > this code is not "onephase".
> >
> > AtEOXact_FdwXact() in fdwxact.c
> > /* One-phase rollback foreign transaction */
> > FdwXactParticipantEndTransaction(fdw_part, false, false);
> >
> > static void
> > FdwXactParticipantEndTransaction(FdwXactParticipant *fdw_part, bool
> > onephase,
> > bool for_commit)
> >
> > * "two_phase_commit" option is mentioned in postgres-fdw.sgml,
> > but I can't find related code.
> >
> > * resolver.c comments have the sentence
> > containing two blanks.(Emergency Termination)
> >
> > * There are some inconsistency with PostgreSQL wiki.
> > https://wiki.postgresql.org/wiki/Atomic_Commit_of_Distributed_Transactions
> >
> > I understand it's difficult to keep consistency, I think it's ok to
> > fix later
> > when these patches almost be able to be committed.
> >
> > - I can't find "two_phase_commit" option in the source code.
> > But 2PC is work if the remote server's "max_prepared_transactions"
> > is set
> > to non zero value. It is correct work, isn't it?
>
> Yes. I had removed two_phase_commit option from postgres_fdw.
> Currently, postgres_fdw uses 2pc when 2pc is required. Therefore,
> max_prepared_transactions needs to be set to more than one, as you
> mentioned.
>
> >
> > - some parameters are renamed or added in latest patches.
> > max_prepared_foreign_transaction, max_prepared_transactions and so
> > on.
> >
> > - typo: froeign_transaction_resolver_timeout
> >
>
> Thank you for your review! I've incorporated your comments on the
> local branch. I'll share the latest version patch.
>
> Also, I've updated the wiki page. I'll try to keep the wiki page up-to-date.
>

I've attached the latest version patches. I've incorporated the review
comments I got so far and improved locking strategy.

Please review it.

Regards,

--
Masahiko Sawada http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachment Content-Type Size
v23-0007-Add-prefer-mode-to-foreign_twophase_commit.patch application/octet-stream 12.3 KB
v23-0006-Add-regression-tests-for-foreign-twophase-commit.patch application/octet-stream 45.2 KB
v23-0005-postgres_fdw-supports-atomic-commit-APIs.patch application/octet-stream 45.9 KB
v23-0004-Documentation-update.patch application/octet-stream 39.5 KB
v23-0003-Support-atomic-commit-among-multiple-foreign-ser.patch application/octet-stream 186.1 KB
v23-0002-Recreate-RemoveForeignServerById.patch application/octet-stream 2.7 KB
v23-0001-Keep-track-of-writing-on-non-temporary-relation.patch application/octet-stream 2.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2020-06-23 03:38:16 Re: Threading in BGWorkers (!)
Previous Message Amit Kapila 2020-06-23 02:48:26 Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions