Re: Synchronous replay take III

From: Michail Nikolaev <michail(dot)nikolaev(at)gmail(dot)com>
To: Michael Paquier <michael(at)paquier(dot)xyz>
Cc: Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>
Subject: Re: Synchronous replay take III
Date: 2018-12-02 20:01:43
Message-ID: CANtu0oh2x+618NVZtGqLoehA0yJATKkNOBShCAc=HdXCCttkXg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello.

It is really nice feature. I am working on the project which heavily reads
from replicas (6 of them).

In our case we have implemented some kind of "replication barrier"
functionality based on table with counters (one counter per application
backend in simple case).
Each application backend have dedicated connection to each replica. And it
selects its counter value few times (2-100) per second from each replica in
background process (depending on how often replication barrier is used).

Once application have committed transaction it may want join replication
barrier before return new data to a user. So, it increments counter in the
table and waits until all replicas have replayed that value according to
background monitoring process. Of course timeout, replicas health checks
and few optimizations and circuit breakers are used.

Nice thing here - constant number of connection involved. Even if lot of
threads joining replication barrier in the moment. Even if some replicas
are lagging.

Because 2-5 seconds lag of some replica will lead to out of connections
issue in few milliseconds in case of implementation described in this
thread.
It may be the weak part of the patch I think. At least for our case. But it
possible could be used to eliminate odd table with counters in my case (if
it possible to change setting per transaction).

Thanks a lot,
Michail.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas Munro 2018-12-02 22:45:00 Re: dsa_allocate() faliure
Previous Message Sergei Agalakov 2018-12-02 18:42:20 Re: {PROPOSAL] add session information column to pg_stat_statements