From: | vignesh C <vignesh21(at)gmail(dot)com> |
---|---|
To: | Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> |
Cc: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> |
Subject: | Re: Replication slot stats misgivings |
Date: | 2021-04-10 07:36:29 |
Message-ID: | CALDaNm1A=bjSrQjBNwNsOtTig+6pZpunmAj_P7Au0H0XjtvCyA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Sat, Apr 10, 2021 at 9:50 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Fri, Apr 9, 2021 at 4:13 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > 2.
> > @@ -2051,6 +2054,17 @@ ReorderBufferProcessTXN(ReorderBuffer *rb,
> > ReorderBufferTXN *txn,
> > rb->begin(rb, txn);
> > }
> >
> > + /*
> > + * Update total transaction count and total transaction bytes, if
> > + * transaction is streamed or spilled it will be updated while the
> > + * transaction gets spilled or streamed.
> > + */
> > + if (!rb->streamBytes && !rb->spillBytes)
> > + {
> > + rb->totalTxns++;
> > + rb->totalBytes += rb->size;
> > + }
> >
> > I think this will skip a transaction if it is interleaved between a
> > streaming transaction. Assume, two transactions t1 and t2. t1 sends
> > changes in multiple streams and t2 sends all changes in one go at
> > commit time. So, now, if t2 is interleaved between multiple streams
> > then I think the above won't count t2.
> >
> > 3.
> > @@ -3524,9 +3538,11 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb,
> > ReorderBufferTXN *txn)
> > {
> > rb->spillCount += 1;
> > rb->spillBytes += size;
> > + rb->totalBytes += size;
> >
> > /* don't consider already serialized transactions */
> > rb->spillTxns += (rbtxn_is_serialized(txn) ||
> > rbtxn_is_serialized_clear(txn)) ? 0 : 1;
> > + rb->totalTxns += (rbtxn_is_serialized(txn) ||
> > rbtxn_is_serialized_clear(txn)) ? 0 : 1;
> > }
> >
> > We do serialize each subtransaction separately. So totalTxns will
> > include subtransaction count as well when serialized, otherwise not.
> > The description of totalTxns also says that it doesn't include
> > subtransactions. So, I think updating rb->totalTxns here is wrong.
> >
>
> The attached patch should fix the above two comments. I think it
> should be sufficient if we just update the stats after processing the
> TXN. We need to ensure that don't count streamed transactions multiple
> times. I have not tested the attached, can you please review/test it
> and include it in the next set of patches if you agree with this
> change.
Thanks Amit for your Patch. I have merged your changes into my
patchset. I did not find any issues in my testing.
Thoughts?
Regards,
Vignesh
Attachment | Content-Type | Size |
---|---|---|
v5-0001-Changed-char-datatype-to-NameData-datatype-for-sl.patch | text/x-patch | 8.7 KB |
v5-0002-Added-total-txns-and-total-txn-bytes-to-replicati.patch | text/x-patch | 19.9 KB |
v5-0003-Added-tests-for-verification-of-logical-replicati.patch | text/x-patch | 5.7 KB |
v5-0004-Handle-overwriting-of-replication-slot-statistic-.patch | text/x-patch | 2.8 KB |
v5-0005-Test-where-there-are-more-replication-slot-statis.patch | text/x-patch | 2.6 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Bharath Rupireddy | 2021-04-10 07:43:37 | Is it worth to optimize VACUUM/ANALYZE by combining duplicate rel instances into single rel instance? |
Previous Message | David Rowley | 2021-04-10 07:20:34 | Re: Lots of incorrect comments in nodeFuncs.c |