Re: kqueue backend for pgbench

From: Kirill Reshke <reshkekirill(at)gmail(dot)com>
To: Roman Khapov <rkhapov(at)yandex-team(dot)ru>
Cc: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: kqueue backend for pgbench
Date: 2026-10-28 05:12:22
Message-ID: CALdSSPg-maGunyKSaqOv-iUEhSfhMS-Kk3i=ZKZ7dJAPnL4pZQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, 26 Oct 2026 at 11:42, Roman Khapov <rkhapov(at)yandex-team(dot)ru> wrote:
>
> Hi hackers!
>
> While working on [1] I run performance tests on different environments,
> including laptop with macOS. pgbench is usually chosen for this,
> but on one of the last runs I discovered that it uses select() on macOS (which lacks
> the ppoll() syscall), which caps the max amount of clients at 1024 (FD_SETSIZE).

Hi! pgbench with this many clients is an uncommon case, but I can see
the benefit in stress-testing connection allocation, which in case of
connection pooler is of interest. So overall +1 for supporting this

> This can be fixed with new implementation of socket_set based on kqueue().
> The only wrinkle is fact that select() and ppoll() is stateless, while kqueue() is not
> (registration persists in kernel across kevent() calls), so wait_on_socket_set() must be
> written in harder way than other implementations:
> - disable stale registrations: EV_DISABLE for every fd to drain error for closed fds
> - enable current set with EV_ENABLE, call kevent()
> - report results (save flags in socket_set->input_ready) + preserving the set for the next iteration
>
> POC-patch attached, any feedback would be appreciated!

Tested on my mac, it works.
Didn't look much in code detail here, but the nested stale-fd loop is
O(nregistered * (num of EV_ERROR)) . Probably fine at pgbench scale.

> [1] - https://github.com/yandex/odyssey
>
> --
> Best regards,
> Roman Khapov

--
Best regards,
Kirill Reshke

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Yura Sokolov 2026-10-28 05:18:38 Re: kqueue backend for pgbench
Previous Message Roman Khapov 2026-10-28 05:04:00 Re: Print pgbench results on SIGINT