Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

On Wed, Sep 30, 2020 at 3:00 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Wed, Sep 30, 2020 at 12:16 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> >
> > On Mon, Sep 28, 2020 at 2:22 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > >
> > >
> > > I don't have a patch for this idea but you can refer [1]
> > > (v25-0002-Issue-individual-invalidations-with-wal_level-lo) to see
> > > what I was trying to say about registering the invalidations in the
> > > form of ReorderBufferChange. We have something along the lines of what
> > > I described above in that patch but we have removed it because we need
> > > all the invalidations to be accumulated to handle aborts and we don't
> > > want two different mechanisms to store invalidations.
> >
> > In some of the intermediate version of the logical-decoding, we had
> > collected in form of changes and later we changed it so that we can
> > execute all the invalidation on abort. I just browsed old patch and
> > fetch the changes to collect the invalidation in form of changes. I
> > have rebased on the current head so that we collect both in form of
> > changes as well as collection of the invalidation. So if we get
> > individiaual invalidation we execute them and on abort we execute all
> > the invalidation together.
> >
>
> Yeah, this is in-line with what I had in mind but please update the
> comments in ReorderBufferAddInvalidations why we need to collect this
> in both the forms. The comments are there but expand them a bit more.

Done

> And you might need to update the below comment as well:
> typedef struct ReorderBufferTXN
> {
> ..
> /*
> * List of ReorderBufferChange structs, including new Snapshots and new
> * CommandIds
> */
> dlist_head changes;
>
> I have tried to think of a way to capture these changes only in one
> form to serve our purpose but couldn't come up with any good ideas but
> if you can think of some idea to achieve that I am all ears.

Even I am not sure of any better way to do this.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com