Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On 01/03/2018 09:06 PM, Tomas Vondra wrote:
> Hi,
>
> attached is v4 of the patch series, with a couple of changes:
>
> 1) Fixes a bunch of bugs I discovered during stress testing.
>
> I'm not going to go into details, but the main fixes are related to
> properly updating progress from the worker, and not streaming when
> creating the logical replication slot.
>
> 2) Introduces columns into pg_stat_replication.
>
> The new columns track various kinds of statistics (number of xacts,
> bytes, ...) about spill-to-disk/streaming. This will be useful when
> tuning the GUC memory limit.
>
> 3) Two temporary bugfixes that make the patch series work.
>

Forgot to mention that the v4 also extends the CREATE SUBSCRIPTION to
allow customizing the streaming and memory limit. So you can do

CREATE SUBSCRIPTION ... WITH (streaming=on, work_mem=1024)

and this subscription will allow streaming, and the logica_work_mem (on
provider) will be set to 1MB.

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services