Re: Inconsistency in reporting checkpointer stats

On Wed, Dec 14, 2022 at 1:02 PM Nitin Jadhav
<nitinjadhavpostgres(at)gmail(dot)com> wrote:
>
> Hi,
>
> While working on checkpoint related stuff, I have encountered that
> there is some inconsistency while reporting checkpointer stats. When a
> checkpoint gets completed, a checkpoint complete message gets logged.
> This message has a lot of information including the buffers written
> (CheckpointStats.ckpt_bufs_written). This variable gets incremented in
> 2 cases. First is in BufferSync() and the second is in
> SlruInternalWritePage(). On the other hand the checkpointer stats
> exposed using pg_stat_bgwriter contains a lot of information including
> buffers written (PendingCheckpointerStats.buf_written_checkpoints).
> This variable gets incremented in only one place and that is in
> BufferSync(). So there is inconsistent behaviour between these two
> data. Please refer to the sample output below.
>
> In order to fix this, the
> PendingCheckpointerStats.buf_written_checkpoints should be incremented
> in SlruInternalWritePage() similar to
> CheckpointStats.ckpt_bufs_written. I have attached the patch for the
> same. Please share your thoughts.

Indeed PendingCheckpointerStats.buf_written_checkpoints needs to count
buffer writes in SlruInternalWritePage(). However, does it need to be
done immediately there? The stats will not be visible to the users
until the next pgstat_report_checkpointer(). Incrementing
buf_written_checkpoints in BufferSync() makes sense as the
pgstat_report_checkpointer() gets called in there via
CheckpointWriteDelay() and it becomes visible to the user immediately.
Have you checked if pgstat_report_checkpointer() gets called while the
checkpoint calls SlruInternalWritePage()? If not, then you can just
assign ckpt_bufs_written to buf_written_checkpoints in
LogCheckpointEnd() like its other friends
checkpoint_write_time and checkpoint_sync_time.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com