Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

From: Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Nathan Bossart <nathandbossart(at)gmail(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, Cary Huang <cary(dot)huang(at)highgo(dot)ca>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: add checkpoint stats of snapshot and mapping files of pg_logical dir
Date: 2022-03-24 09:52:11
Message-ID: CALj2ACXvTbJLBVcaN5DxQ-tThOKMBeFNmYoyh_8gdAkL1wyHNw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Mar 24, 2022 at 8:58 AM Bharath Rupireddy
<bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
>
> On Wed, Mar 23, 2022 at 10:16 AM Bharath Rupireddy
> <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
> >
> > On Tue, Mar 22, 2022 at 8:12 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> > > > Do you mean like this?
> > > > ereport(LOG,
> > > > /* translator: the placeholders show checkpoint options */
> > > > (errmsg("%s starting:%s%s%s%s%s%s%s%s",
> > > > restartpoint ? _("restartpoint") : _("checkpoint"),
> > > > (flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
> > > > (flags & CHECKPOINT_END_OF_RECOVERY) ? "
> > > > end-of-recovery" : "",
> > > > (flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
> > > > (flags & CHECKPOINT_FORCE) ? " force" : "",
> > > > (flags & CHECKPOINT_WAIT) ? " wait" : "",
> > > > (flags & CHECKPOINT_CAUSE_XLOG) ? " wal" : "",
> > > > (flags & CHECKPOINT_CAUSE_TIME) ? " time" : "",
> > > > (flags & CHECKPOINT_FLUSH_ALL) ? " flush-all" : "")));
> > >
> > > Yes.
> >
> > Done that way, see
> > v7-0001-Deduplicate-checkpoint-restartpoint-starting-comp.patch.
> >
> > > > I think the reason in this case might be that some flag names with hyphens
> > > > and spaces before words may not have the right/matching words in all
> > > > languages. What happens if we choose to translate/not translate the entire
> > > > message?
> > >
> > > If individual words aren't translated the "original" word would be used.
> >
> > Interestingly, the translated message for "checkpoint/restart
> > complete" is empty. Maybe because it has untranslatable strings?
> >
> > #: access/transam/xlog.c:8752
> > #, c-format
> > msgid "restartpoint complete: wrote %d buffers (%.1f%%); %d WAL
> > file(s) added, %d removed, %d recycled; write=%ld.%03d s,
> > sync=%ld.%03d s, total=%ld.%03d s; sync files=%d, longest=%ld.%03d s,
> > average=%ld.%03d s; distance=%d kB, estimate=%d kB"
> > msgstr ""
> >
> > > > > Both seem still very long. I still am doubtful this level of detail is
> > > > > appropriate. Seems more like a thing for a tracepoint or such. How about just
> > > > > printing the time for the logical decoding operations in aggregate, without
> > > > > breaking down into files, adding LSNs etc?
> > > >
> > > > The distinction that the patch makes right now is for snapshot and
> > > > rewrite mapping files and it makes sense to have them separately.
> > >
> > > -1. The line also needs to be readable...
> >
> > IMHO, that's subjective. Even now, the existing
> > "checkpoint/restartpoint complete" message has a good amount of info
> > which makes it unreadable for some.
> >
> > The number of logical decoding files(snapshot and mapping) the
> > checkpoint processed is a good metric to have in server logs along
> > with the time it took for removing/syncing them. Thoughts?
>
> Realized that the CF bot is applying patches in the alphabetical order
> (I've sent out both v7 patches as v7-0001). Attaching v8 patch-set
> with v8-0001 and v8-0002 names. Apart from this, no change in v8.

CF bot is complaining
https://cirrus-ci.com/task/6458067389775872?logs=mingw_cross_warning#L399.

Attaching v9 patch-set to fix it.

Regards,
Bharath Rupireddy.

Attachment Content-Type Size
v9-0001-Deduplicate-checkpoint-restartpoint-starting-comp.patch application/octet-stream 5.7 KB
v9-0002-Add-checkpoint-stats-of-snapshot-and-mapping-file.patch application/octet-stream 7.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dilip Kumar 2022-03-24 10:02:01 Re: Support logical replication of DDLs
Previous Message Bharath Rupireddy 2022-03-24 09:32:29 Re: pg_walinspect - a new extension to get raw WAL data and WAL stats