Re: Resetting spilled txn statistics in pg_stat_replication

On Sun, 21 Jun 2020 at 06:57, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com> wrote:
>
> On Thu, Jun 18, 2020 at 12:21:17PM +0530, Amit Kapila wrote:
> >On Thu, Jun 18, 2020 at 8:01 AM Masahiko Sawada
> ><masahiko(dot)sawada(at)2ndquadrant(dot)com> wrote:
> >>
> >> On Wed, 17 Jun 2020 at 20:14, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >> >
> >> >
> >> > I had written above in the context of persisting these stats. I mean
> >> > to say if the process has bounced or server has restarted then the
> >> > previous stats might not make much sense because we were planning to
> >> > use pid [1], so the stats from process that has exited might not make
> >> > much sense or do you think that is okay? If we don't want to persist
> >> > and the lifetime of these stats is till the process is alive then we
> >> > are fine.
> >> >
> >>
> >> Sorry for confusing you. The above my idea is about having the stats
> >> per slots. That is, we add spill_txns, spill_count and spill_bytes to
> >> pg_replication_slots or a new view pg_stat_logical_replication_slots
> >> with some columns: slot_name plus these stats columns and stats_reset.
> >> The idea is that the stats values accumulate until either the slot is
> >> dropped, the server crashed, the user executes the reset function, or
> >> logical decoding is performed with different logical_decoding_work_mem
> >> value than the previous time. In other words, the stats values are
> >> reset in either case. That way, I think the stats values always
> >> correspond to logical decoding using the same slot with the same
> >> logical_decoding_work_mem value.
> >>
> >
> >What if the decoding has been performed by multiple backends using the
> >same slot? In that case, it will be difficult to make the judgment
> >for the value of logical_decoding_work_mem based on stats. It would
> >make sense if we provide a way to set logical_decoding_work_mem for a
> >slot but not sure if that is better than what we have now.
> >

I thought that the stats are relevant to what
logical_decoding_work_mem value was but not with who performed logical
decoding. So even if multiple backends perform logical decoding using
the same slot, the user can directly use stats as long as
logical_decoding_work_mem value doesn’t change.

> >What problems do we see in displaying these for each process? I think
> >users might want to see the stats for the exited processes or after
> >server restart but I think both of those are not even possible today.
> >I think the stats are available till the corresponding WALSender
> >process is active.

I might want to see the stats for the exited processes or after server
restart. But I'm inclined to agree with displaying the stats per
process if the stats are displayed on a separate view (e.g.
pg_stat_replication_slots).

> >
>
> I don't quite see what the problem is. We're in this exact position with
> many other stats we track and various GUCs. If you decide to tune the
> setting for a particular slot, you simply need to be careful which
> backends decode the slot and what GUC values they used.
>
> But I don't think this situation (multiple backends decoding the same
> slot with different logical_decoding_work_mem values) is very common. In
> most cases the backends/walsenders will all use the same value. If you
> change that, you better remember that.
>
> I really think we should not be inventing something that automatically
> resets the stats when someone happens to change the GUC.

Agreed. But what I thought is more simple; storing the
logical_decoding_work_mem value along with the stats into a logical
replication slot and resetting the stats if the
logical_decoding_work_mem value is different than the stored value
when performing logical decoding.

Regards,

--
Masahiko Sawada http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services