| From: | Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com> |
|---|---|
| To: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Cc: | Andres Freund <andres(at)anarazel(dot)de> |
| Subject: | aio: Async fsyncs for crash recovery and checkpointer |
| Date: | 2026-08-25 12:48:15 |
| Message-ID: | CAN55FZ0vLWJQNB=HuHXG2wabFjXJd6OWTa3=kRzwObdZD9poHQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
This patch implements async fsyncs for crash recovery and checkpoints
by using the AIO system. I got the idea from Andres' older work [1].
----------
High level design and implementation details:
- Since there is no interface like read-stream for the fsyncs, each
user has its own AIO functions like register_one(), drain_one() and
drain_all().
- Number of in-flight I/Os are determined by
GetFsyncConcurrencyLimit(). It uses both io_max_concurrency and the
file descriptor budget to determine max number of available in-flight
fsync I/Os.
- I/O workers need re-open files to do async I/O. Some targets open
files by path (for example, SyncDataDirectory()). This path needs to
be copied to shared memory so that worker processes can re-open these
files. However, adding 'char path[MAXPGPATH]' to PgAioTargetData seems
overkill because of the size; so it is not implemented for now. I am
open to suggestions.
----------
Patches:
- 0001 is just a base patch for adding fsync support to the AIO
system. There is no user of this patch yet.
- 0002 makes SyncDataDirectory() use async fsyncs. Its files are
identified only by path; I/O workers cannot re-open them. So, this
path is asynchronous with io_uring, while worker behaves like sync.
pre_sync_fname() doesn't use AIO because it is already very fast, less
than 1/1000 of of datadir_fsync_fname() time in my benchmarks.
- 0003 makes ProcessSyncRequests() use async fsyncs. Its files use the
smgr target and can be re-opened by an I/O worker, so both worker and
io_uring can execute these fsyncs asynchronously.
- 0004 adds a FileTag-based target for SLRU files. This gives I/O
workers enough information to re-open SLRU segments, so they can use
async fsyncs in the worker mode.
- benchmark-repro.sh is a benchmark script that I used.
----------
Benchmark:
I used a simple benchmark script generated by an LLM. I checked the
script, and it works correctly. The numbers below use the default
settings + the following:
```
checkpoint_timeout = '1h'
max_wal_size = '32GB'
checkpoint_flush_after = 0
bgwriter_lru_maxpages = 0
autovacuum = off
```
io_min_workers, io_max_workers and io_max_concurrency are default in
[2]. They are changed to 8, 8, and 64, respectively, for the second
benchmark [3].
You can run the attached benchmark script by './benchmark-repro.sh
{--syncdatadirectory, --checkpointer, --slru}' command.
- SyncDataDirectory() synced a data directory containing 4000 small relations:
[2]
io_method | startup (ms) | speedup vs sync
-----------+----------------------+----------------
sync | 1411.8 | 1.00x
worker | 1210.6 | 1.17x
io_uring | 310.3 | 4.55
[3]
io_method | startup (ms) | speedup vs sync
-----------+----------------------+----------------
sync | 1310.6 | 1.00x
worker | 1210.0 | 1.08x
io_uring | 310.6 | 4.22x
- Checkpointer benchmark dirtied 4000 relation files before checkpoint:
[2]
io_method | checkpoint (ms) | speedup vs sync
-----------+----------------------+----------------
sync | 1860.5 | 1.00x
worker | 697.1 | 2.67x
io_uring | 478.4 | 3.89x
[3]
io_method | checkpoint (ms) | speedup vs sync
-----------+----------------------+----------------
sync | 1875.5 | 1.00x
worker | 541.1 | 3.47x
io_uring | 479.9 | 3.91x
- SLRU benchmark created 640000 transactions, producing ~30 files to sync:
[2]
io_method | SLRU checkpoint (ms) | speedup vs sync
-----------+----------------------+----------------
sync | 94.1 | 1.00x
worker | 67.1 | 1.40x
io_uring | 57.1 | 1.65x
[3]
io_method | SLRU checkpoint (ms) | speedup vs sync
-----------+----------------------+----------------
sync | 97.3 | 1.00x
worker | 48.7 | 2.00x
io_uring | 37.4 | 2.60x
Any feedback would be appreciated.
[1]
https://github.com/anarazel/postgres/commit/9829a1f18176e572759ecf8b8bb99205a71a476f
https://github.com/anarazel/postgres/commit/2e180547aef6fe2a409cc144e04cbc2f0a607bed
--
Regards,
Nazir Bilal Yavuz
Microsoft
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0001-aio-Add-fsync-support.patch | text/x-patch | 5.9 KB |
| v1-0002-Issue-SyncDataDirectory-fsyncs-through-AIO.patch | text/x-patch | 18.9 KB |
| v1-0003-Issue-checkpointer-fsyncs-asynchronously.patch | text/x-patch | 38.9 KB |
| v1-0004-Allow-IO-workers-to-execute-SLRU-fsyncs.patch | text/x-patch | 20.5 KB |
| benchmark-repro.sh | application/x-shellscript | 3.6 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Shlok Kyal | 2026-08-25 12:51:01 | Re: Apply worker can pick an invalid index for REPLICA IDENTITY FULL lookups |
| Previous Message | Sagar Shedge | 2026-08-25 12:48:00 | [PATCH v1] Report specific wait events for libpq cancel requests |