Re: Skipping logical replication transactions on subscriber side

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
Cc: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, "tanghy(dot)fnst(at)fujitsu(dot)com" <tanghy(dot)fnst(at)fujitsu(dot)com>, "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com>, Alexey Lesovsky <lesovsky(at)gmail(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Greg Nancarrow <gregn4422(at)gmail(dot)com>
Subject: Re: Skipping logical replication transactions on subscriber side
Date: 2021-09-25 07:23:11
Message-ID: CAA4eK1J3Fc0=tE2ZR7XnEfeZm52_4hyQZO1s7kEALFLvZsXxRQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Sep 24, 2021 at 6:44 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Fri, Sep 24, 2021 at 8:01 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> >
> > 6.
> > +typedef struct PgStat_StatSubEntry
> > +{
> > + Oid subid; /* hash table key */
> > +
> > + /*
> > + * Statistics of errors that occurred during logical replication. While
> > + * having the hash table for table sync errors we have a separate
> > + * statistics value for apply error (apply_error), because we can avoid
> > + * building a nested hash table for table sync errors in the case where
> > + * there is no table sync error, which is the common case in practice.
> > + *
> >
> > The above comment is not clear to me. Why do you need to have a
> > separate hash table for table sync errors? And what makes it avoid
> > building nested hash table?
>
> In the previous patch, a subscription stats entry
> (PgStat_StatSubEntry) had one hash table that had error entries of
> both apply and table sync. Since a subscription can have one apply
> worker and multiple table sync workers it makes sense to me to have
> the subscription entry have a hash table for them.
>

Sure, but each tablesync worker must have a separate relid. Why can't
we have a single hash table for both apply and table sync workers
which are hashed by sub_id + rel_id? For apply worker, the rel_id will
always be zero (InvalidOId) and tablesync workers will have a unique
OID for rel_id, so we should be able to uniquely identify each of
apply and table sync workers.

--
With Regards,
Amit Kapila.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Fabien COELHO 2021-09-25 07:40:50 Re: rand48 replacement
Previous Message Amit Kapila 2021-09-25 06:46:40 Re: POC: Cleaning up orphaned files using undo logs