RE: Failed transaction statistics to measure the logical replication progress

From: "wangw(dot)fnst(at)fujitsu(dot)com" <wangw(dot)fnst(at)fujitsu(dot)com>
To: "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com>
Cc: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, "vignesh21(at)gmail(dot)com" <vignesh21(at)gmail(dot)com>, "amit(dot)kapila16(at)gmail(dot)com" <amit(dot)kapila16(at)gmail(dot)com>, "sawada(dot)mshk(at)gmail(dot)com" <sawada(dot)mshk(at)gmail(dot)com>, "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, 'Greg Nancarrow' <gregn4422(at)gmail(dot)com>
Subject: RE: Failed transaction statistics to measure the logical replication progress
Date: 2021-12-23 09:37:02
Message-ID: OS3PR01MB627528A542EFDBDED8DC7F289E7E9@OS3PR01MB6275.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wednesday, December 22, 2021 10:30 PM
osumi(dot)takamichi(at)fujitsu(dot)com <osumi(dot)takamichi(at)fujitsu(dot)com> wrote:
> On Wednesday, December 22, 2021 8:38 PM I wrote:
> > Do we expect these commit counts which come from empty transactions ?
> This is another issue discussed in [1]
> where the patch in the thread is a work in progress, I think.
> ......
> IMHO, the conclusion is we are currently in the middle of fixing the behavior.

Thank you for telling me this.
After applying v19-* and v15-0001-Skip-empty-transactions-for-logical-replication.patch,
I retested v19-* patches. The result of previous case looks good to me.

But the results of following cases are also similar to previous unexpected result
which increases commit_count or abort_count unexpectedly.
[1]
(Based on environment in the previous example, set TWO_PHASE=true)
[Publisher]
begin;
insert into replica_test1 values(1,'1');
prepare transaction 'id';
commit prepared 'id';

In subscriber side, the commit_count of two records(sub1 and sub2) is increased.

[2]
(Based on environment in the previous example, set STREAMING=on)
[Publisher]
begin;
INSERT INTO replica_test1 SELECT i, md5(i::text) FROM generate_series(1, 5000) s(i);
commit;

In subscriber side, the commit_count of two records(sub1 and sub2) is increased.

[3]
(Based on environment in the previous example, set TWO_PHASE=true)
[Publisher]
begin;
insert into replica_test1 values(1,'1');
prepare transaction 'id';
rollback prepared 'id';

In subscriber side, the abort_count of two records(sub1 and sub2) is increased.

I think the problem maybe is the patch you mentioned
(Skip-empty-transactions-for-logical-replication.patch) is not finished yet.
Share this information here.

Regards,
Wang wei

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bharath Rupireddy 2021-12-23 10:47:06 Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes
Previous Message Masahiko Sawada 2021-12-23 08:44:37 Re: Add index scan progress to pg_stat_progress_vacuum