Re: Replication slot stats misgivings

On Wed, Apr 28, 2021 at 3:25 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Wed, Apr 28, 2021 at 9:37 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> >
> > On Wed, Apr 28, 2021 at 12:29 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > >
> > > On Tue, Apr 27, 2021 at 11:02 AM vignesh C <vignesh21(at)gmail(dot)com> wrote:
> > > >
> > > > On Tue, Apr 27, 2021 at 9:48 AM vignesh C <vignesh21(at)gmail(dot)com> wrote:
> > > > >
> > > >
> > > > Attached patch has the changes to update statistics during
> > > > spill/stream which prevents the statistics from being lost during
> > > > interrupt.
> > > >
> > >
> > > void
> > > -UpdateDecodingStats(LogicalDecodingContext *ctx)
> > > +UpdateDecodingStats(ReorderBuffer *rb)
> > >
> > > I don't think you need to change this interface because
> > > reorderbuffer->private_data points to LogicalDecodingContext. See
> > > StartupDecodingContext.
> >
> > +1
> >
> > With this approach, we could still miss the totalTxns and totalBytes
> > updates if the decoding a large but less than
> > logical_decoding_work_mem is interrupted, right?
> >
>
> Right, but is there some simple way to avoid that? I see two
> possibilities (a) store stats in ReplicationSlot and then send them at
> ReplicationSlotRelease but that will lead to an increase in shared
> memory usage and as per the discussion above, we don't want that, (b)
> send intermediate stats after decoding say N changes but for that, we
> need to additionally compute the size of each change which might add
> some overhead.

Right.

> I am not sure if any of these alternatives are a good idea. What do
> you think? Do you have any other ideas for this?

I've been considering some ideas but don't come up with a good one
yet. It’s just an idea and not tested but how about having
CreateDecodingContext() register before_shmem_exit() callback with the
decoding context to ensure that we send slot stats even on
interruption. And FreeDecodingContext() cancels the callback.

Regards,

--
Masahiko Sawada
EDB: https://www.enterprisedb.com/