Re: postmaster uses more CPU in 18 beta1 with io_method=io_uring

From: Andres Freund <andres(at)anarazel(dot)de>
To: Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>
Cc: Robert Treat <rob(at)xzilla(dot)net>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Jim Nasby <jnasby(at)upgrade(dot)com>, "Burd, Greg" <greg(at)burd(dot)me>, MARK CALLAGHAN <mdcallag(at)gmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Tomas Vondra <tomas(at)vondra(dot)me>, Nathan Bossart <nathandbossart(at)gmail(dot)com>
Subject: Re: postmaster uses more CPU in 18 beta1 with io_method=io_uring
Date: 2025-09-21 16:29:30
Message-ID: ei54ndz27kl53uvhz7r5lntxtio54imrrbnoqt3rwa3nv7yhvw@qwzgzeg7oivp
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

> From ad7c856e964b614507a06342c2acbf10bfa4855c Mon Sep 17 00:00:00 2001
> From: Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>
> Date: Tue, 9 Sep 2025 14:30:48 +0200
> Subject: [PATCH v1] aio: warn user if combined io_uring memory mappings are
> unavailable
>
> In f54af9f2 we have added solution to avoid connection and disconnection hit
> caused by io_uring managing large number of memory mappings. Unfortunately
> it is available only on more modern Linux kernels (6.5) therefore notify user
> in visible way if this optimization is not available.
>
> Author: Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>
> Reviewed-by:
> Discussion: https://postgr.es/m/CAFbpF8OA44_UG+RYJcWH9WjF7E3GA6gka3gvH6nsrSnEe9H0NA@mail.gmail.com
> ---
> doc/src/sgml/config.sgml | 6 ++++++
> src/backend/storage/aio/method_io_uring.c | 14 ++++++++++----
> 2 files changed, 16 insertions(+), 4 deletions(-)
>
> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
> index 2a3685f474a..9d541999dc1 100644
> --- a/doc/src/sgml/config.sgml
> +++ b/doc/src/sgml/config.sgml
> @@ -2784,6 +2784,12 @@ include_dir 'conf.d'
> <para>
> This parameter can only be set at server start.
> </para>
> + <para>
> + Note that for optimum performance with <literal>io_uring</literal>
> + Linux kernel version >= 6.5 is recommended, as it provides way to
> + reduce the number of additional memory mappings which may negatively
> + affect the efficiency of establishing and terminating connections.
> + </para>
> </listitem>
> </varlistentry>

This seems too low-level for end user docs, while not explaining that the
impact is due to a high max_connections value, rather than a large number of
actually established connections. How about something like

Note that for optimal performance with <literal>io_uring</literal> Linux
kernel version >= 6.5 is recommended. Older Linux versions, high values
of <xref linkend="guc-max-connections"/> will slow down connection
establishment and termination.

> diff --git a/src/backend/storage/aio/method_io_uring.c b/src/backend/storage/aio/method_io_uring.c
> index bb06da63a8e..5cd839df2f3 100644
> --- a/src/backend/storage/aio/method_io_uring.c
> +++ b/src/backend/storage/aio/method_io_uring.c
> @@ -207,8 +207,11 @@ pgaio_uring_check_capabilities(void)
> * pgaio_uring_shmem_init().
> */
> errno = -ret;
> - elog(DEBUG1,
> - "cannot use combined memory mapping for io_uring, ring creation failed: %m");
> + ereport(WARNING,
> + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
> + errmsg("cannot use combined memory mapping for io_uring, ring creation failed: %m"),
> + errdetail("Connection and disconnection rates and efficiency may be degraded."),
> + errhint("Ensure that you are running kernel >= 6.5")));

To me this seems too verbose, particularly because the majority of users
encountering it have zero chance to address the issue. And it's not like most
real world workloads are particularly affected, if you run with
max_connections=20k and have 100/connections second, you'll have a *lot* of
other problems.

Here's the full log of a start with the fallback branch forced:

2025-09-21 12:20:49.666 EDT [4090828][postmaster][:0][] WARNING: cannot use combined memory mapping for io_uring, ring creation failed: Unknown error -8192
2025-09-21 12:20:49.666 EDT [4090828][postmaster][:0][] DETAIL: Connection and disconnection rates and efficiency may be degraded.
2025-09-21 12:20:49.666 EDT [4090828][postmaster][:0][] HINT: Ensure that you are running kernel >= 6.5
2025-09-21 12:20:49.708 EDT [4090828][postmaster][:0][] LOG: starting PostgreSQL 19devel on x86_64-linux, compiled by gcc-15.2.0, 64-bit
2025-09-21 12:20:49.708 EDT [4090828][postmaster][:0][] LOG: listening on IPv6 address "::1", port 5440
2025-09-21 12:20:49.708 EDT [4090828][postmaster][:0][] LOG: listening on IPv4 address "127.0.0.1", port 5440
2025-09-21 12:20:49.708 EDT [4090828][postmaster][:0][] LOG: listening on Unix socket "/tmp/.s.PGSQL.5440"
2025-09-21 12:20:49.712 EDT [4090831][startup][:0][] LOG: database system was shut down at 2025-09-21 12:20:42 EDT
2025-09-21 12:20:49.717 EDT [4090828][postmaster][:0][] LOG: database system is ready to accept connections

Close to half the lines are the new warning.

Greetings,

Andres Freund

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jim Jones 2025-09-21 16:42:28 Re: Add notification on BEGIN ATOMIC SQL functions using temp relations
Previous Message Jim Jones 2025-09-21 15:37:06 Re: Add notification on BEGIN ATOMIC SQL functions using temp relations