Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Simon Riggs <simon(at)2ndquadrant(dot)com>
Cc: Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Keisuke Kuroda <keisuke(dot)kuroda(dot)3862(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables
Date: 2020-10-09 09:04:54
Message-ID: CAA4eK1JMtFFN6VsFaF9+S2MsjPwkfqxvWFyTqmQe6+5Hh+sXvQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Oct 9, 2020 at 2:19 PM Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>
> On Fri, 9 Oct 2020 at 04:10, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Thu, Oct 8, 2020 at 2:34 PM Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> > >
> > > On Thu, 8 Oct 2020 at 09:47, Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> > >
> > > > > This script will wait 10 seconds after INSERT exits
> > > > > before executing TRUNCATE, please wait for it to run.
> > >
> > > Has this been tested with anything other than the one test case?
> > >
> > > It would be good to know how the patch handles a transaction that
> > > contains many aborted subtransactions that contain invals.
> > >
> >
> > Are you thinking from the angle of performance or functionality? I
> > don't see how this patch can impact either of those. Basically, it
> > will not execute any extra invalidations then it is executing without
> > the patch for aborted subtransactions. Can you please explain in a bit
> > more detail about your fear?
> >
> > Having said that, I think it would be a good idea to test the scenario
> > you mentioned to ensure that we have not broken anything unknowingly.
>
> The test appears to only cover the case of many subtransactions, all
> of which commit, and then top-level commit occurs.
>
> We should be testing cases where the top-level commit occurs, yet some
> proportion of the subtransactions abort. "Normal" would be 10-50%
> aborts.
>
> I presume we support this case already, but wish to ensure the
> performance tweak is not just for the one special case.
>

Okay, I think this makes sense. I think we should see the performance
benefit for this case as well but maybe to a bit lesser degree because
we will exclude some of the subtransactions from processing.

--
With Regards,
Amit Kapila.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Greg Nancarrow 2020-10-09 09:06:33 Re: Parallel INSERT (INTO ... SELECT ...)
Previous Message Simon Riggs 2020-10-09 08:49:24 Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables