From 99b4f39ce43b2b34f51a2fb64f72e5d70d14c305 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot Date: Mon, 26 Oct 2026 13:50:58 +0000 Subject: [PATCH v2 1/8] Update buffer strategy descriptions in the buffer README The buffer README still describes the bulk-read strategy as using a fixed 256kB ring. Commit 15f0cb26b530 changed this but forgot to update the README: the requested ring size now starts at 256kB and grows to accommodate the configured I/O concurrency, subject to the backend pin limit and the general cap of one eighth of shared_buffers. The actual ring can therefore be smaller than 256kB. The README also says that the bulk-write strategy applies only to COPY IN and CREATE TABLE AS SELECT. It is now used by additional operations, including materialized view population and some table rewrites. Update both descriptions. Author: Bertrand Drouvot Reviewed-by: Sami Imseih Discussion: https://postgr.es/m/auCYYbDyrQEwIsQq@bdtpg --- src/backend/storage/buffer/README | 41 ++++++++++++++++++------------- 1 file changed, 24 insertions(+), 17 deletions(-) 100.0% src/backend/storage/buffer/ diff --git a/src/backend/storage/buffer/README b/src/backend/storage/buffer/README index b332e002ba1..430a9ba7a3c 100644 --- a/src/backend/storage/buffer/README +++ b/src/backend/storage/buffer/README @@ -216,19 +216,26 @@ whole scan. This also implies that much of the write traffic caused by such a statement will be done by the backend itself and not pushed off onto other processes. -For sequential scans, a 256KB ring is used. That's small enough to fit in L2 -cache, which makes transferring pages from OS cache to shared buffer cache -efficient. Even less would often be enough, but the ring must be big enough -to accommodate all pages in the scan that are pinned concurrently. 256KB -should also be enough to leave a small cache trail for other backends to -join in a synchronized seq scan. If a ring buffer is dirtied and its LSN -updated, we would normally have to write and flush WAL before we could -re-use the buffer; in this case we instead discard the buffer from the ring -and (later) choose a replacement using the normal clock-sweep algorithm. -Hence this strategy works best for scans that are read-only (or at worst -update hint bits). In a scan that modifies every page in the scan, like a -bulk UPDATE or DELETE, the buffers in the ring will always be dirtied and -the ring strategy effectively degrades to the normal strategy. +For bulk reads, the requested ring size is at least 256KB. The 256KB target is +small enough to fit in L2 cache, which makes transferring pages from OS cache +to shared buffer cache efficient. Even less would often be enough, but the +target must be big enough to accommodate all pages in the scan that are +pinned concurrently. The 256KB target should also be enough to leave a small +cache trail for other backends to join in a synchronized seq scan. + +The requested ring size is enlarged to accommodate io_combine_limit * +effective_io_concurrency buffers while I/O is in progress, but no further +than the backend's pin limit allows. Like all strategy rings, the actual size +is capped at 1/8th of shared_buffers and can therefore be less than 256KB. + +If a ring buffer is dirtied and its LSN updated, we would normally have to +write and flush WAL before we could re-use the buffer; in this case we instead +discard the buffer from the ring and (later) choose a replacement using the +normal clock-sweep algorithm. Hence this strategy works best for scans that +are read-only (or at worst update hint bits). In a scan that modifies every +page in the scan, like a bulk UPDATE or DELETE, the buffers in the ring will +always be dirtied and the ring strategy effectively degrades to the normal +strategy. VACUUM uses a ring like sequential scans, however, the size of this ring is controlled by the vacuum_buffer_usage_limit GUC. Dirty pages are not removed @@ -237,10 +244,10 @@ buffers. Before introducing the buffer ring strategy in 8.3, VACUUM's buffers were sent to the freelist, which was effectively a buffer ring of 1 buffer, resulting in excessive WAL flushing. -Bulk writes work similarly to VACUUM. Currently this applies only to -COPY IN and CREATE TABLE AS SELECT. (Might it be interesting to make -seqscan UPDATE and DELETE use the bulkwrite strategy?) For bulk writes -we use a ring size of 16MB (but not more than 1/8th of shared_buffers). +Bulk writes work similarly to VACUUM. They are used for operations such as +COPY FROM, CREATE TABLE AS, materialized view population, and some table +rewrites. For bulk writes we use a ring size of 16MB (but not more than +1/8th of shared_buffers). Smaller sizes have been shown to result in the COPY blocking too often for WAL flushes. While it's okay for a background vacuum to be slowed by doing its own WAL flushing, we'd prefer that COPY not be subject to that, -- 2.34.1