RE: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

From: "kuroda(dot)hayato(at)fujitsu(dot)com" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, 'Fujii Masao' <masao(dot)fujii(at)oss(dot)nttdata(dot)com>
Subject: RE: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
Date: 2021-11-11 02:27:28
Message-ID: TYAPR01MB58668C69BA6CC0FEA79D7336F5949@TYAPR01MB5866.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dear Fujita-san,

I love your proposal because it will remove a bottleneck
for PostgreSQL build-in sharding.

I read your patch briefly and I think basically it's good.
Currently I have only one comment.

In your patch, postgres_fdw sends a COMMIT command to all entries in the hash table
and waits for the result without a timeout from the first entry.
I think this specification is good because it's very simple,
but if a COMMIT for a particular foreign server could take some time,
I thought it might be more efficient to stop waiting for results and look at the next entry.
This is how it works. First, we define a function similar to pgfdw_get_result()
so that we can specify the timeout time as an argument to WaitLatchOrSocket().
Then change the function called by do_sql_command_end () to the new one,
and change the callback function to skip if the result has not yet arrived

How is it? Is it an unnecessary assumption that COMMIT takes time? Or is this the next step?
I will put a PoC if needed.

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2021-11-11 02:40:36 Re: parallel vacuum comments
Previous Message Tom Lane 2021-11-11 02:20:28 Re: Frontend error logging style