eXtensible Transaction Manager API (v2)

From: Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: eXtensible Transaction Manager API (v2)
Date: 2016-02-10 17:50:56
Message-ID: 56BB7880.4020604@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

PostgresProffesional cluster teams wants to propose new version of
eXtensible Transaction Manager API.
Previous discussion concerning this patch can be found here:

http://www.postgresql.org/message-id/F2766B97-555D-424F-B29F-E0CA0F6D1D74@postgrespro.ru

The API patch itself is small enough, but we think that it will be
strange to provide just API without examples of its usage.

We have implemented various implementations of distributed transaction
manager based on this API:
pg_dtm (based ion snapshot sharing) and pg_tsdtm (CSN based on local
system time).
Based on this two DTM implementation we have developed various "cluster"
implementations:
multimaster+pg_dtm, multimaster+pg_tsdtm, pg_shard+pg_dtm,
pg_shard+pg_tsdtm, postgres_fdw+pg_dtm, postgres_fdw+pg+tsdtm,...
Multimaster is based on logical replication is something like BDR but
synchronous: provide consistency across cluster.

But we want to make this patch as small as possible.
So we decided to include in it only pg_tsdtm and patch of postgres_fdw
allowing to use it with pg_tsdtm.
pg_tsdtm is simpler than pg_dtm because last one includes arbiter with
RAFT protocol (centralized service)
and sockhub for efficient multiplexing backend connections.
Also, in theory, pg_tsdtm provides better scalability, because it is
decentralized.

Architecture of DTM and tsDTM as well as benchmark results are available
at WiKi page:

https://wiki.postgresql.org/wiki/DTM

Please notice pg-tsdtm is just reference implementation of DTM using
this XTM API.
The primary idea of this patch is to add XTM API to PostgreSQL code,
allowing to implement own transaction managers as
Postgres extension. So please review first of all XTM API itself and not
pg_tsdtm which is just and example of its usage.

The complete PostgreSQL branch with all our changes can be found here:

https://github.com/postgrespro/postgres_cluster

-- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachment Content-Type Size
pg_tsdtm.patch text/x-patch 37.1 KB
postgres_fdw.patch text/x-patch 20.4 KB
xtm.patch text/x-patch 19.4 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2016-02-10 17:53:28 Re: Tracing down buildfarm "postmaster does not shut down" failures
Previous Message Anastasia Lubennikova 2016-02-10 17:46:48 Some refactoring of index structures .