aio tests failing on newer Linux kernels

From: Alexander Lakhin <exclusion(at)gmail(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Cc: Andres Freund <andres(at)anarazel(dot)de>
Subject: aio tests failing on newer Linux kernels
Date: 2026-07-26 20:00:00
Message-ID: 667f7dcf-404b-4898-a426-57ca6e0617f6@gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello hackers,

Multiple failures produced recently by debian animals, e.g. [1], [2], [3]:
pgsql.build/testrun/test_aio/001_aio/log/regress_log_001_aio
...
[23:22:28.951](0.000s) ok 280 - worker: failure to open: recovers: expected stderr
### Stopping node "worker" using mode fast
# Running: pg_ctl --pgdata
/home/bf/bf-build/tamandua/HEAD/pgsql.build/testrun/test_aio/001_aio/data/t_001_aio_worker_data/pgdata --mode fast stop
waiting for server to shut down.... done
server stopped
# No postmaster PID for node "worker"
### Starting node "io_uring"
# Running: pg_ctl --wait --pgdata
/home/bf/bf-build/tamandua/HEAD/pgsql.build/testrun/test_aio/001_aio/data/t_001_aio_io_uring_data/pgdata --log
/home/bf/bf-build/tamandua/HEAD/pgsql.build/testrun/test_aio/001_aio/log/001_aio_io_uring.log --options
--cluster-name=io_uring start
waiting for server to start.... stopped waiting
pg_ctl: could not start server
Examine the log output.
# pg_ctl start failed; see logfile for details:
/home/bf/bf-build/tamandua/HEAD/pgsql.build/testrun/test_aio/001_aio/log/001_aio_io_uring.log
# No postmaster PID for node "io_uring"
[23:22:29.992](1.040s) Bail out!  pg_ctl start failed

pgsql.build/testrun/test_aio/001_aio/log/001_aio_io_uring.log
...
2026-07-25 23:22:29.949 CEST [2396766][postmaster][:0] FATAL:  could not setup io_uring queue: Cannot allocate memory
...

show a new anomaly, probably caused by a kernel update. I can't see in
the test log, which kernels the animals use, but I reproduced the same
failure locally, on debian sid, having multiplied test/modules/test_aio
x30 and executed:
$ meson test test_aio_* -j 30
...
 12/120 test_aio_2 - postgresql:test_aio_2/004_read_stream            OK 10.26s   31 subtests passed
 13/120 test_aio_5 - postgresql:test_aio_5/004_read_stream            OK 10.41s   31 subtests passed
 14/120 test_aio_7 - postgresql:test_aio_7/003_initdb                 ERROR 11.75s   (exit status 255 or 0xff)
...
119/120 test_aio_28 - postgresql:test_aio_28/001_aio                  OK 31.15s   826 subtests passed
120/120 test_aio_30 - postgresql:test_aio_30/003_initdb               OK 26.77s   6 subtests passed

Ok:                93
Fail:              27

$ grep -r 'could not setup io_uring queue: Cannot allocate memory' testrun/ | wc -l
27

(The tests pass for me with -j 10.)

This is reproduced on
Linux debian-13 6.19.6+deb14-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.19.6-1 (2026-03-06) x86_64 GNU/Linux
but not reproduced on:
Linux debian-13 6.12.48+deb13-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.12.48-1 (2025-09-20) x86_64

AI suggests the new behavior is caused by the "io_uring: use region api
for SQ/CQ" change landed in the kernel version 6.14.

[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=kestrel&dt=2026-07-25%2010%3A13%3A20
[2] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=melonworm&dt=2026-07-25%2017%3A18%3A24
[3] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tamandua&dt=2026-07-25%2021%3A15%3A22

Best regards,
Alexander

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2026-07-26 20:34:43 Re: remove_useless_joins vs. bug #19560
Previous Message Peter Geoghegan 2026-07-26 19:14:48 Simplifying how nbtree represents array scan keys