aio: Async fsyncs for crash recovery and checkpointer

From: Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Andres Freund <andres(at)anarazel(dot)de>
Subject: aio: Async fsyncs for crash recovery and checkpointer
Date: 2026-08-25 12:48:15
Message-ID: CAN55FZ0vLWJQNB=HuHXG2wabFjXJd6OWTa3=kRzwObdZD9poHQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

This patch implements async fsyncs for crash recovery and checkpoints
by using the AIO system. I got the idea from Andres' older work [1].

----------

High level design and implementation details:

- Since there is no interface like read-stream for the fsyncs, each
user has its own AIO functions like register_one(), drain_one() and
drain_all().

- Number of in-flight I/Os are determined by
GetFsyncConcurrencyLimit(). It uses both io_max_concurrency and the
file descriptor budget to determine max number of available in-flight
fsync I/Os.

- I/O workers need re-open files to do async I/O. Some targets open
files by path (for example, SyncDataDirectory()). This path needs to
be copied to shared memory so that worker processes can re-open these
files. However, adding 'char path[MAXPGPATH]' to PgAioTargetData seems
overkill because of the size; so it is not implemented for now. I am
open to suggestions.

----------

Patches:

- 0001 is just a base patch for adding fsync support to the AIO
system. There is no user of this patch yet.

- 0002 makes SyncDataDirectory() use async fsyncs. Its files are
identified only by path; I/O workers cannot re-open them. So, this
path is asynchronous with io_uring, while worker behaves like sync.
pre_sync_fname() doesn't use AIO because it is already very fast, less
than 1/1000 of of datadir_fsync_fname() time in my benchmarks.

- 0003 makes ProcessSyncRequests() use async fsyncs. Its files use the
smgr target and can be re-opened by an I/O worker, so both worker and
io_uring can execute these fsyncs asynchronously.

- 0004 adds a FileTag-based target for SLRU files. This gives I/O
workers enough information to re-open SLRU segments, so they can use
async fsyncs in the worker mode.

- benchmark-repro.sh is a benchmark script that I used.

----------

Benchmark:

I used a simple benchmark script generated by an LLM. I checked the
script, and it works correctly. The numbers below use the default
settings + the following:

```
checkpoint_timeout = '1h'
max_wal_size = '32GB'
checkpoint_flush_after = 0
bgwriter_lru_maxpages = 0
autovacuum = off
```
io_min_workers, io_max_workers and io_max_concurrency are default in
[2]. They are changed to 8, 8, and 64, respectively, for the second
benchmark [3].

You can run the attached benchmark script by './benchmark-repro.sh
{--syncdatadirectory, --checkpointer, --slru}' command.

- SyncDataDirectory() synced a data directory containing 4000 small relations:

[2]
io_method | startup (ms) | speedup vs sync
-----------+----------------------+----------------
sync | 1411.8 | 1.00x
worker | 1210.6 | 1.17x
io_uring | 310.3 | 4.55

[3]
io_method | startup (ms) | speedup vs sync
-----------+----------------------+----------------
sync | 1310.6 | 1.00x
worker | 1210.0 | 1.08x
io_uring | 310.6 | 4.22x

- Checkpointer benchmark dirtied 4000 relation files before checkpoint:

[2]
io_method | checkpoint (ms) | speedup vs sync
-----------+----------------------+----------------
sync | 1860.5 | 1.00x
worker | 697.1 | 2.67x
io_uring | 478.4 | 3.89x

[3]
io_method | checkpoint (ms) | speedup vs sync
-----------+----------------------+----------------
sync | 1875.5 | 1.00x
worker | 541.1 | 3.47x
io_uring | 479.9 | 3.91x

- SLRU benchmark created 640000 transactions, producing ~30 files to sync:

[2]
io_method | SLRU checkpoint (ms) | speedup vs sync
-----------+----------------------+----------------
sync | 94.1 | 1.00x
worker | 67.1 | 1.40x
io_uring | 57.1 | 1.65x

[3]
io_method | SLRU checkpoint (ms) | speedup vs sync
-----------+----------------------+----------------
sync | 97.3 | 1.00x
worker | 48.7 | 2.00x
io_uring | 37.4 | 2.60x

Any feedback would be appreciated.

[1]
https://github.com/anarazel/postgres/commit/9829a1f18176e572759ecf8b8bb99205a71a476f
https://github.com/anarazel/postgres/commit/2e180547aef6fe2a409cc144e04cbc2f0a607bed

--
Regards,
Nazir Bilal Yavuz
Microsoft

Attachment Content-Type Size
v1-0001-aio-Add-fsync-support.patch text/x-patch 5.9 KB
v1-0002-Issue-SyncDataDirectory-fsyncs-through-AIO.patch text/x-patch 18.9 KB
v1-0003-Issue-checkpointer-fsyncs-asynchronously.patch text/x-patch 38.9 KB
v1-0004-Allow-IO-workers-to-execute-SLRU-fsyncs.patch text/x-patch 20.5 KB
benchmark-repro.sh application/x-shellscript 3.6 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Shlok Kyal 2026-08-25 12:51:01 Re: Apply worker can pick an invalid index for REPLICA IDENTITY FULL lookups
Previous Message Sagar Shedge 2026-08-25 12:48:00 [PATCH v1] Report specific wait events for libpq cancel requests