Re: Skipping logical replication transactions on subscriber side

On Mon, Nov 1, 2021 at 7:18 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Fri, Oct 29, 2021 at 8:20 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> >
> > Fair enough. So statistics can be removed either by vacuum or drop
> > subscription. Also, if we go by this logic then there is no harm in
> > retaining the stat entries for tablesync errors. Why have different
> > behavior for apply and tablesync workers?
>
> My understanding is that the subscription worker statistics entry
> corresponds to workers (but not physical workers since the physical
> process is changed after restarting). So if the worker finishes its
> jobs, it is no longer necessary to show errors since further problems
> will not occur after that. Table sync worker’s job finishes when
> completing table copy (unless table sync is performed again by REFRESH
> PUBLICATION) whereas apply worker’s job finishes when the subscription
> is dropped.
>

Actually, I am not very sure how users can use the old error
information after we allowed skipping the conflicting xid. Say, if
they want to add/remove some constraints on the table based on
previous errors then they might want to refer to errors of both the
apply worker and table sync worker.

> Also, I’m concerned about a situation like where a lot of
> table sync failed. In which case, if we don’t drop table sync worker
> statistics after completing its job, we end up having a lot of entries
> in the view unless the subscription is dropped.
>

True, but the same could be said for apply workers where errors can be
accumulated over a period of time.

> >
> > I have another question in this regard. Currently, the reset function
> > seems to be resetting only the first stat entry for a subscription.
> > But can't we have multiple stat entries for a subscription considering
> > the view's cumulative nature?
>
> I might be missing your points but I think that with the current
> patch, the view has multiple entries for a subscription. That is,
> there is one apply worker stats and multiple table sync worker stats
> per subscription.
>

Can't we have multiple entries for one apply worker?

> And pg_stat_reset_subscription() function can reset
> any stats by specifying subscription OID and relation OID.
>

Say, if the user has supplied just subscription OID then isn't it
better to reset all the error entries for that subscription?

--
With Regards,
Amit Kapila.