From: | Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> |
---|---|
To: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> |
Cc: | "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: Failed transaction statistics to measure the logical replication progress |
Date: | 2021-09-29 11:55:07 |
Message-ID: | CAA4eK1KzZxCOPqGPiRj0s=xxtgyVv+6Wm-yRoEccnmMV489ZNw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Wed, Sep 29, 2021 at 4:49 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Tue, Sep 28, 2021 at 7:04 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Tue, Sep 28, 2021 at 11:35 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > >
> > > On Tue, Sep 28, 2021 at 1:54 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > > >
> > > > >
> > > > > Then, if, we proceed in this direction,
> > > > > the place to implement those stats
> > > > > would be on the LogicalRepWorker struct, instead ?
> > > > >
> > > >
> > > > Or, we can make existing stats persistent and then add these stats on
> > > > top of it. Sawada-San, do you have any thoughts on this matter?
> > >
> > > I think that making existing stats including received_lsn and
> > > last_msg_receipt_time persistent by using stats collector could cause
> > > massive reporting messages. We can report these messages with a
> > > certain interval to reduce the amount of messages but we will end up
> > > seeing old stats on the view.
> > >
> >
> > Can't we keep the current and new stats both in-memory and persist on
> > disk? So, the persistent stats data will be used to fill the in-memory
> > counters after restarting of workers, otherwise, we will always refer
> > to in-memory values.
>
> Interesting. Probably we can have apply workers and table sync workers
> send their statistics to the stats collector at exit (before the stats
> collector shutting down)? And the startup process will restore them at
> restart?
>
I think we do need to send at the exit but we should probably send at
some other regular interval as well to avoid losing all the stats
after the crash-restart situation.
--
With Regards,
Amit Kapila.
From | Date | Subject | |
---|---|---|---|
Next Message | Teodor Sigaev | 2021-09-29 12:14:42 | Re: On login trigger: take three |
Previous Message | Amit Kapila | 2021-09-29 11:52:39 | Re: Some thoughts about the TAP tests' wait_for_catchup() |