Re: Automatically sizing the IO worker pool

> On Sat, Apr 11, 2026 at 06:35:18PM +1200, Thomas Munro wrote:
> On Wed, Jul 30, 2025 at 10:15 PM Dmitry Dolgov <9erthalion6(at)gmail(dot)com> wrote:
> > As a side note, I was trying to experiment with this patch using
> > dm-mapper's delay feature to introduce an arbitrary large io latency and
> > see how the io queue is growing.
>
> FWIW, here's what I came up with while experimenting with that sort of thing:
>
> shared_preload_libraries=io_limit
> io_limit.ios_per_second=6000
>
> That differs from eg dm-mapper delays by making everything seem like
> slow direct I/O, which seemed more interesting for this project. For
> example if you run some continuous workload while you SET
> io_limit.ios_per_second to various numbers, with
> io_workers_idle_timeout set fairly low, you can monitor the pool
> adjustments.

Yeah, sounds like a good idea. Do you plan to introduce such an
extension long term for testing, or is it just one off?

As to me it looks worth keeping, maybe even use injections points to
allow for more flexibility. And I know I sound like a broken record, but
if I understand correctly the delays introduced via ios_per_seconds and
others are constant in time -- I've experimented a bit and found some
reference implementations in numpy for geometric distribution sampling,
which allow to make the delay a random variable. Since the geometric
distribution is a discrete analog of the exponential one, and the latter
represents delays between events in Poisson distribution, such random
variable would give an approximation for more real load.