May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

From: "Anton A(dot) Melnikov" <aamelnikov(at)inbox(dot)ru>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.
Date: 2022-09-06 11:02:53
Message-ID: 99b2ccd1-a77a-962a-0837-191cdf56c2b9@inbox.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello!

Found a periodic spike growth of the checkpoint_req counter on replica by 20-30 units
after large insert (~350Mb) on master.
Reproduction on master and replica with default conf:
1) execute the command "insert into test values (generate_series(1,1E7));".
This leads to the table's growth by about 350Mb during about 15 secs (on my pc).
2)The wal records start coming to the replica, and when their number exceeds a certain limit, a request is emitted to the checkpointer process to create restartpoint on the replica and checkpoint_req is incremented. With default settings, this limit is 42 segments.
3) Restartpoint creation fails because a new restartpoint can only be created if the replica has received new WAL records about the checkpoint from the moment of the previous restartpoint. But there were no such records.
4) When the next WAL segment is received by replica, the next request is generated to create a restartpoint on the replica, and so on.
5) Finally, a WAL record about the checkpoint arrives on the replica, restartpoint is created and the growth of checkpoint_req stops.
The described process can be observed in the log with additional debugging. See insert_1E7_once.log attached. This
log is for v13 but master has the same behavior.

Can we treat such behavior as a bug?
If so it seems possible to check if a creating of restartpoint is obviously impossible before sending request and don't send it at all if so.

The patch applied tries to fix it.

With best regards.
--
Anton A. Melnikov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachment Content-Type Size
insert_1E7_once.log text/x-log 14.6 KB
v1-0001-Fix-burst-checkpoint_req-growth.patch text/x-patch 2.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2022-09-06 11:13:14 Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher
Previous Message Himanshu Upadhyaya 2022-09-06 10:34:10 Re: HOT chain validation in verify_heapam()