Re: libpq compression

From: "Andrey M(dot) Borodin" <x4mmm(at)yandex-team(dot)ru>
To: Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: libpq compression
Date: 2020-10-06 06:34:14
Message-ID: 74F0A385-6ED4-4EA7-9498-1A8527F16472@yandex-team.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> 26 марта 2019 г., в 19:46, Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru> написал(а):
>
> Version of the patch correctly working when no compression algorithm are avaiable.

Thanks for this work, Konstantin.
PFA rebased version of this patch.

This compression seems very important to reduce network-induced replication lag.
Recently I've found out that small installation suffer from huge latency spike when archive_timeout occurs.
How this happens? archive_timeout emits segment switch, which pads current segment with zeroes. Now these zeroes need to be replicated to standbys, without compression. Transfer of zeroes causes long waits for synchronous replication (up to 1s with network restricted to 16Mbps).

Also replication compression will reduce overall cross-AZ traffic of HA installations.

So I'm considering following plan for 2020: implement this protocol in Odyssey and send patches to drivers to enable early access to this feature.

Best regards, Andrey Borodin.

Attachment Content-Type Size
v16-0001-libpq-compression.patch application/octet-stream 44.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bharath Rupireddy 2020-10-06 06:44:22 Re: Logical Replication - detail message with names of missing columns
Previous Message Bharath Rupireddy 2020-10-06 06:11:18 Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module