From: | Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> |
---|---|
To: | Nathan Bossart <nathandbossart(at)gmail(dot)com> |
Cc: | Michael Paquier <michael(at)paquier(dot)xyz>, Cary Huang <cary(dot)huang(at)highgo(dot)ca>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: add checkpoint stats of snapshot and mapping files of pg_logical dir |
Date: | 2022-03-16 09:32:26 |
Message-ID: | CALj2ACVnhe=8=QtRPGo3PJRvh4TmSb3Ebdm=_QR+NDCT=rxQkg@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Wed, Mar 16, 2022 at 1:47 AM Nathan Bossart <nathandbossart(at)gmail(dot)com> wrote:
>
> On Tue, Mar 15, 2022 at 11:04:26AM +0900, Michael Paquier wrote:
> > On Mon, Mar 14, 2022 at 03:54:19PM +0530, Bharath Rupireddy wrote:
> >> At times, the snapshot or mapping files can be large in number and one
> >> some platforms it takes time for checkpoint to process all of them.
> >> Having the stats about them in server logs can help us better analyze
> >> why checkpoint took a long time and provide a better RCA.
> >
> > Do you have any numbers to share regarding that? Seeing information
> > about 1k WAL segments being recycled and/or removed by a checkpoint
> > where the operation takes dozens of seconds to complete because we can
> > talk about hundred of gigs worth of files moved around. If we are
> > talking about 100~200 files up to 10~20kB each for snapshot and
> > mapping files, the information has less value, worth only a portion of
> > one WAL segment.
>
> I don't have specific numbers to share, but as noted elsewhere [0], I
> routinely see lengthy checkpoints that spend a lot of time in these cleanup
> tasks.
>
> [0] https://postgr.es/m/18ED8B1F-7F5B-4ABF-848D-45916C938BC7%40amazon.com
I too don't have numbers in hand to share. But I have come across
cases where checkpoints spend a good amount of time cleaning
up/processing snapshots or mapping files.
Regards,
Bharath Rupireddy.
From | Date | Subject | |
---|---|---|---|
Next Message | Bharath Rupireddy | 2022-03-16 09:32:41 | Re: add checkpoint stats of snapshot and mapping files of pg_logical dir |
Previous Message | Fabien COELHO | 2022-03-16 09:28:57 | Re: [PATCH] pgbench: add multiconnect option |