Re: New statistics for tuning WAL buffer size

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Masahiro Ikeda <ikedamsh(at)oss(dot)nttdata(dot)com>
Cc: Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com>, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)fujitsu(dot)com>, Magnus Hagander <magnus(at)hagander(dot)net>
Subject: Re: New statistics for tuning WAL buffer size
Date: 2020-09-28 02:41:23
Message-ID: CAA4eK1KqiH=+NZn8Z4BsaQLG71azD0vzPQqbYB4X_uUsJDR5Fw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Sep 28, 2020 at 7:00 AM Masahiro Ikeda <ikedamsh(at)oss(dot)nttdata(dot)com> wrote:
>
> On 2020-09-26 19:18, Amit Kapila wrote
>
> > This makes sense to me. I think even if such background processes have
> > to write WAL due to wal_buffers, it will be accounted next time the
> > backend sends the stats.
>
> Thanks for your comments.
>
> IIUC, since each process counts WalStats.m_wal_buffers_full,
> backend can't send the counter which other background processes have to
> write WAL due to wal_buffers.
>

Right, I misunderstood it.

> Although we can't track all WAL activity, the impact on the statistics
> is minimal so we can ignore it.
>

Yeah, that is probably true.

> > One minor point, don't we need to reset the counter
> > WalStats.m_wal_buffers_full once we sent the stats, otherwise the same
> > stats will be accounted multiple times.
>
> Now, the counter is reset in pgstat_send_wal.
> Isn't it enough?
>

That should be enough.

One other thing that occurred to me today is can't we keep this as
part of PgStat_GlobalStats? We can use pg_stat_reset_shared('wal'); to
reset it. It seems to me this is a cluster-wide stats and somewhat
similar to some of the other stats we maintain there.

--
With Regards,
Amit Kapila.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2020-09-28 02:46:16 Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables
Previous Message Tom Lane 2020-09-28 02:35:06 Re: Small improvements to pg_list.h's linitial(), lsecond(), lthird() etc macros