[PATCH] Use streaming read I/O in sample scans

From: Yuhang Qiu <iamqyh(at)gmail(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: [PATCH] Use streaming read I/O in sample scans
Date: 2026-08-26 13:45:41
Message-ID: D1F99EB2-5A03-465F-A210-98F7DF256168@gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers,

SampleScan currently reads each block selected by the sampling method
with `ReadBufferExtended()`. This prevents it from issuing reads for
later pages while processing the current one. The early TABLESAMPLE
discussion also mentioned using `NextSampleBlock()` for prefetching [1].

This patch uses a ReadStream to obtain buffers for SampleScan. For
sampling methods that provide `NextSampleBlock()`, the callback supplies
blocks to the stream. Other methods keep the existing sequential scan
behavior. The stream is created after `SampleScanState` becomes available,
then reset and reused on rescan.

The other two patches handle buffer access strategy changes during
rescan and report SampleScan I/O statistics in
`EXPLAIN (ANALYZE, IO)`.

I tested this with a 64 GiB table containing 8,388,608 pages, with one
row per page. PostgreSQL was restarted before each run to clear shared
buffers. The main settings were:
debug_io_direct=data
io_method=worker
shared_buffers=128MB
effective_io_concurrency=16

Results:
master patch speedup
SYSTEM(0.1) 829ms 121ms 6.8x
SYSTEM(1) 7.53s 0.89s 8.4x
SYSTEM(10) 70.84s 10.62s 6.7x
SYSTEM_ROWS(100000) 9.05s 0.97s 9.3x
BERNOULLI(100) LIMIT 100000 1.83s 0.56s 3.3x

With 8 concurrent clients running `SYSTEM(1)`, throughput increased
from 1.06 TPS to 4.29 TPS.

Performance was almost unchanged with `effective_io_concurrency=0`.
With worker I/O and `effective_io_concurrency=1`, the patch was about
8% slower. When all data was cached, the regression was about 1% to 2%.
Queries that stop early, such as `LIMIT` and `SYSTEM_ROWS`, may also
issue a small number of unused speculative reads.

`SYSTEM_TIME` needs some explanation. It currently checks the time only
in `NextSampleBlock()`, so the existing implementation is not a strict
time limit and may exceed the requested time while processing the
current page. A ReadStream may request a limited number of blocks in
advance, so the deviation can grow to the look-ahead window. This patch
does not add special handling for `SYSTEM_TIME`.

The regression tests, `tsm_system_rows`, `tsm_system_time`, cassert,
rescans, and fixed-seed comparisons for `SYSTEM` and `BERNOULLI` all
passed. `SYSTEM_ROWS` still returns exactly the requested number of rows.

Besides SampleScan, I found several paths that still read blocks one at
a time without using ReadStream:

- `ProcessSingleRelationFork()` while enabling online checksums;
- `log_newpage_range()`, which reads a block range while generating WAL;
- exact heap and index scans in pgstattuple;
- nbtree and GIN verification in amcheck;
- index-driven heap fetch, which has also been discussed separately [2].

These paths have different access patterns and constraints. Do people
think any of them should be converted to ReadStream?

There is also a broader question. Some in-core pathes and many external
extensions still call `ReadBuffer()` directly, and some higher-level
workloads have predictable block access patterns that are not exposed to
the buffer manager. Adapting individual core paths cannot cover all of
these cases.

It may be worth discussing whether DIO needs a more general heuristic
read-ahead mechanism. If there is interest, I can start a separate thread.
That is outside the scope of this patch.

Best Regards,
Yuhang Qiu

[1] https://www.postgresql.org/message-id/flat/12048.1436646520%40sss.pgh.pa.us

[2] https://www.postgresql.org/message-id/flat/f3xxfrkafjxpyqxywcxricxgyizjirfceychyxsgn7bwjp5eda%40kwbduhy7tfmu

Attachment Content-Type Size
0001-read_stream-Allow-changing-the-buffer-access-strateg.patch application/octet-stream 2.7 KB
0002-heapam-Use-streaming-read-I-O-in-sample-scans.patch application/octet-stream 8.1 KB
0003-explain-Report-streaming-I-O-statistics-for-sample-s.patch application/octet-stream 4.0 KB
unknown_filename text/plain 2 bytes

Browse pgsql-hackers by date

  From Date Subject
Next Message Bertrand Drouvot 2026-08-26 13:49:24 Persist slot invalidations before publishing them
Previous Message Amit Langote 2026-08-26 13:42:47 Re: scary patch contest