Re: Logical replication existing data copy

On 27/02/17 11:07, Erik Rijkers wrote:
> With these patches:
>
> -- 0416d87c-09a5-182e-4901-236aec103e9f(at)2ndquadrant(dot)com
> Subject: Re: Logical Replication WIP
> 48.
> https://www.postgresql.org/message-id/attachment/49886/0001-Use-asynchronous-connect-API-in-libpqwalreceiver.patch
>
> 49.
> https://www.postgresql.org/message-id/attachment/49887/0002-Fix-after-trigger-execution-in-logical-replication.patch
>
> 50.
> https://www.postgresql.org/message-id/attachment/49888/0003-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION.patch
>
>
> -- 51f65289-54f8-2256-d107-937d662d69f1(at)2ndquadrant(dot)com
> Subject: Re: snapbuild woes
> 48.
> https://www.postgresql.org/message-id/attachment/49995/snapbuild-v4-0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch
>
> 49.
> https://www.postgresql.org/message-id/attachment/49996/snapbuild-v4-0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch
>
> 50.
> https://www.postgresql.org/message-id/attachment/49997/snapbuild-v4-0003-Prevent-snapshot-builder-xmin-from-going-backwards.patch
>
> 51.
> https://www.postgresql.org/message-id/attachment/49998/snapbuild-v4-0004-Fix-xl_running_xacts-usage-in-snapshot-builder.patch
>
> 52.
> https://www.postgresql.org/message-id/attachment/49999/snapbuild-v4-0005-Skip-unnecessary-snapshot-builds.patch
>
>
> -- c0f90176-efff-0770-1e79-0249fb4b9b0c(at)2ndquadrant(dot)com
> Subject: Re: Logical replication existing data copy
> 48.
> https://www.postgresql.org/message-id/attachment/49977/0001-Logical-replication-support-for-initial-data-copy-v6.patch
>
>
>
> logical replication now seems pretty stable, at least for the limited
> testcase that I am using. I've done dozens of pgbench_derail2.sh runs
> without failure. I am now changing the pgbench-test to larger scale
> (pgbench -is) and longer periods (-T) which makes running the test slow
> (both instances are running on a modest desktop with a single 7200
> disk). It is quite a bit slower than I expected (a 5-minute pgbench
> scale 5, with 8 clients, takes, after it has finished on master, another
> 2-3 minutes to get synced on the replica). I suppose it's just a
> hardware limitation. I set max_sync_workers_per_subscription to 6 (from
> default 2) but it doesn't help much (at all).
>
> To be continued...
>

Thanks,

The performance was why in original patch I wanted the apply process to
default to synchronous_commit = off as without it the apply performance
(due to applying transactions individually and in sequences) is quite
lackluster.

It can be worked around using user that has synchronous_commit = off set
via ALTER ROLE as owner of the subscription.

--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services