Re: New statistics for tuning WAL buffer size

On 2020-09-18 11:11, Kyotaro Horiguchi wrote:
> At Fri, 18 Sep 2020 09:40:11 +0900, Masahiro Ikeda
> <ikedamsh(at)oss(dot)nttdata(dot)com> wrote in
>> Thanks. I confirmed that it causes HOT pruning or killing of
>> dead index tuple if DecodeCommit() is called.
>>
>> As you said, DecodeCommit() may access the system table.
> ...
>> The wals are generated only when logical replication is performed.
>> So, I added pgstat_send_wal() in XLogSendLogical().
>>
>> But, I concerned that it causes poor performance
>> since pgstat_send_wal() is called per wal record,
>
> I think that's too frequent. If we want to send any stats to the
> collector, it is usually done at commit time using
> pgstat_report_stat(), and the function avoids sending stats too
> frequently. For logrep-worker, apply_handle_commit() is calling it. It
> seems to be the place if we want to send the wal stats. Or it may be
> better to call pgstat_send_wal() via pgstat_report_stat(), like
> pg_stat_slru().

Thanks for your comments.
Since I changed to use pgstat_report_stat() and DecodeCommit() is
calling it,
the frequency to send statistics is not so high.

> Currently logrep-laucher, logrep-worker and autovac-launcher (and some
> other processes?) don't seem (AFAICS) sending scan stats at all but
> according to the discussion here, we should let such processes send
> stats.

I added pgstat_report_stat() to logrep-laucher and autovac-launcher.
As you said, logrep-worker already calls apply_handle_commit() and
pgstat_report_stat().

The checkpointer doesn't seem to call pgstat_report_stat() currently,
but since there is a possibility to send wal statistics, I added
pgstat_report_stat().

Regards,
--
Masahiro Ikeda
NTT DATA CORPORATION