Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Cc: Simon Riggs <simon(at)2ndquadrant(dot)com>, Keisuke Kuroda <keisuke(dot)kuroda(dot)3862(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables
Date: 2020-10-14 10:42:20
Message-ID: CAA4eK1Ki56ZoaUMX4zRqxRnUVTn8CR7EJ_x3_8vJUnO6FuaStQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Oct 12, 2020 at 3:23 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Fri, Oct 9, 2020 at 2:34 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> >
> > Okay, I think this makes sense. I think we should see the performance
> > benefit for this case as well but maybe to a bit lesser degree because
> > we will exclude some of the subtransactions from processing.
>
> I have tried with a combination of abort/commit subtransaction and I
> could see a similar benefit with the patch.
>
> I tested below transaction
> BEGIN;
> truncate table nsp_001.tbl_001;
> savepoint s1;
> truncate table nsp_001.tbl_001;
> savepoint s2;
> truncate table part_0001;
> savepoint s3;
> truncate table part_0002;
> savepoint s5;
> truncate table part_0003;
> rollback to s3;
> commit;
> EOF
>

Thanks for the tests. The latest patch looks mostly good to me. I have
made minor changes to the patch (a) changed the order where the new
message is placed at one place to make it consistent with other
places, (b) as discussed offlist, removed the extra increment to a
local variable in ReorderBufferRestoreChange, (c) ran pgindent.

See the attached and let me know what do you think?

--
With Regards,
Amit Kapila.

Attachment Content-Type Size
v5-0001-Execute-invalidation-messages-for-each-XLOG_XACT_.patch application/octet-stream 10.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dilip Kumar 2020-10-14 11:21:24 Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables
Previous Message Masahiko Sawada 2020-10-14 10:11:49 Re: Transactions involving multiple postgres foreign servers, take 2