Re: Time based lag tracking for logical replication

On Wed, May 03, 2017 at 08:28:53AM +0200, Simon Riggs wrote:
> On 23 April 2017 at 01:10, Petr Jelinek <petr(dot)jelinek(at)2ndquadrant(dot)com> wrote:
> > Hi,
> >
> > The time based lag tracking commit [1] added interface for logging
> > progress of replication so that we can report lag as time interval
> > instead of just bytes. But the patch didn't contain patch for the
> > builtin logical replication.
> >
> > So I wrote something that implements this. I didn't like all that much
> > the API layering in terms of exporting the walsender's LagTrackerWrite()
> > for use by plugin directly. Normally output plugin does not have to care
> > if it's running under walsender or not, it uses abstracted write
> > interface for that which can be implemented in various ways (that's how
> > we implement SQL interface to logical decoding after all). So I decided
> > to add another function to the logical decoding write api called
> > update_progress and call that one from the output plugin. The walsender
> > then implements that new API to call the LagTrackerWrite() while the SQL
> > interface just does not implement it at all. This seems like cleaner way
> > of doing it.
> >
> > Thoughts?
>
> Agree cleaner.
>
> I don't see any pacing or comments about it, nor handling of
> intermediate messages while we process a large transaction.
>
> I'll look at adding some pacing code in WalSndUpdateProgress()

[Action required within three days.]

Simon, I understand, from an annotation on the open items list, that you have
volunteered to own this item. Please observe the policy on open item
ownership[1] and send a status update within three calendar days of this
message. Include a date for your subsequent status update. Testers may
discover new open items at any time, and I want to plan to get them all fixed
well in advance of shipping v10. Consequently, I will appreciate your efforts
toward speedy resolution. Thanks.

[1] https://www.postgresql.org/message-id/20170404140717.GA2675809%40tornado.leadboat.com