Re: Checkpointer write combining

From: Soumya S Murali <soumyamurali(dot)work(at)gmail(dot)com>
To: Melanie Plageman <melanieplageman(at)gmail(dot)com>
Cc: Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>, Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Andres Freund <andres(at)anarazel(dot)de>
Subject: Re: Checkpointer write combining
Date: 2026-01-23 07:03:40
Message-ID: CAMtXxw-2xFGrFzQ7O_9_a0zSJytkh6v-se5JvroCCQXtXUt=VA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi all,

Thank you all for the patches.
I am keeping this as a single patch because the refactoring, batching
behavior and instrumentation are tightly coupled and all serve one
purpose to reduce checkpoint writeback overhead while making the
effect observable. Due to version and context differences, the patches
did not apply cleanly in my development environment. Instead, I
studied the patches and went through the logic in detail and then
implemented the same ideas directly in my current tree adapting them
wherever needed. The implementation was then validated with
instrumentation and measurements.

Before batching:
2026-01-22 17:27:26.969 IST [148738] LOG: checkpoint complete: wrote
15419 buffers (94.1%), wrote 1 SLRU buffers; 0 WAL file(s) added, 0
removed, 25 recycled; write=0.325 s, sync=0.284 s, total=0.754 s; sync
files=30, longest=0.227 s, average=0.010 s; distance=407573 kB,
estimate=407573 kB; lsn=0/1A5B8E30, redo lsn=0/1A5B8DD8

After batching:
2026-01-22 17:31:36.165 IST [148738] LOG: checkpoint complete: wrote
13537 buffers (82.6%), wrote 1 SLRU buffers; 0 WAL file(s) added, 0
removed, 25 recycled; write=0.260 s, sync=0.211 s, total=0.625 s; sync
files=3, longest=0.205 s, average=0.070 s; distance=404310 kB,
estimate=407247 kB; lsn=0/3308E738, redo lsn=0/3308E6E0

Debug instrumentation with (batch size = 16) confirms the batching
behavior itself,
buffers_written = 6196
writeback_calls = 389
On average: I am getting 15.9 i.e approx 16 buffers per writeback
This shows that writebacks are issued per batch rather than per
buffer, while WAL ordering and durability semantics remain unchanged.
The change remains localized to BufferSync() and is intended to be a
conservative and measurable improvement to checkpoint I/O behavior. I
am attaching the patches herewith for review.
I am happy to adjust the approach if there are concerns or
suggestions. Looking forward to more feedback.

Regards,
Soumya

Attachment Content-Type Size
0001-Batch-buffer-writebacks-during-checkpoints.patch text/x-patch 3.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2026-01-23 07:03:59 Some tests for TOAST, STORAGE MAIN/EXTENDED
Previous Message shveta malik 2026-01-23 07:02:47 Re: pg_upgrade: optimize replication slot caught-up check