Re: Report bytes and transactions actually sent downtream

On Wed, Sep 24, 2025 at 10:12 AM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:
>
> I tested the flows with
> a) logical replication slot and get-changes.
> b) filtered data flows: pub-sub creation with row_filters, 'publish'
> options. I tried to verify plugin fields as compared to total_wal*
> fields.
> c) reset flow.
>
> While tests for a and c are present already. I don't see tests for b
> anywhere when it comes to stats. Do you think we shall add a test for
> filtered data using row-filter somewhere?

Added a test in 028_row_filter. Please find it in the attached
patchset. I didn't find tests which test table level filtering or
operation level filtering. Can you please point me to such tests. I
will add similar test to other places. Once you review the test in
028_row_filter, I will replicate it to other places you point out.

On Wed, Sep 24, 2025 at 12:12 PM Bertrand Drouvot
<bertranddrouvot(dot)pg(at)gmail(dot)com> wrote:
>
> Hi,
>
> On Wed, Sep 24, 2025 at 11:38:30AM +0530, shveta malik wrote:
> > On Wed, Sep 24, 2025 at 11:08 AM Ashutosh Bapat
> > <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
> > >
> > > In WalSndWriteData() we can't rely on what happens in a low level API
> > > like socket_putmessage(). And we are counting the number of bytes in
> > > the logically decoded message. So, I actually wonder whether we should
> > > count 1 byte of 'd' in sentBytes. Shveta, Bertand, what do you think?
> > >
> >
> > If we are not counting all such metadata bytes ((or can't reliably do
> > so), then IMO, we shall skip counting msgtype as well.
>
> Agree. Maybe mention in the doc that metadata (including msgtype) bytes are not
> taken into account?

We are counting the sentBytes in central places through which all the
logically decoded messages flow. So we are not missing on any metadata
bytes. Given that these bytes are part of the logically decoded
message itself, I think we should count them in the sentBytes. Now the
question remains is whether to count 4 bytes for length in the message
itself? The logical decoding code can not control that and thus should
not account for it. So I am leaving bytes counted for
pg_hton32((uint32) (len + 4)) out of sentBytes calculation.
--
Best Wishes,
Ashutosh Bapat