Re: BUG #19622: io_method=worker retains file descriptors on dropped relations

From: Alexandre Felipe <o(dot)alexandre(dot)felipe(at)gmail(dot)com>
To: sarangsai(at)gmail(dot)com, pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: Re: BUG #19622: io_method=worker retains file descriptors on dropped relations
Date: 2026-09-20 20:35:48
Message-ID: CAE8JnxODigfggrahaYTpuioz7pvzUGSSZh23kDeb_rCgHPma_A@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hi SeongHan,

I submitted a patch fixing it:
https://commitfest.postgresql.org/patch/7322/

Thank you for the extensive explanation

On Thu, Sep 17, 2026 at 8:37 PM PG Bug reporting form <
noreply(at)postgresql(dot)org> wrote:

> The following bug has been logged on the website:
>
> PostgreSQL version: 18.6
>

It seems unlikely that this will be backpatched.

> Operating system: Debian Linux aarch64 (official postgres:18 Docker)
> Description:
>
> SUMMARY
> =======
>
> With io_method=worker (the default in PostgreSQL 18), AIO I/O worker
> processes retain open file descriptors on relation files that have been
> dropped. After the files are unlinked -- for the first segment this
> happens at the first checkpoint after DROP -- /proc/<pid>/fd shows io
> workers still holding descriptors on them:
>
...

> Regular backends do not accumulate them: they receive SMGR invalidation
> via the shared invalidation queue and call smgrclose(). io workers are
> auxiliary processes and never register as invalidation receivers, so
> that message never reaches them.
>

>
> This is the answer.

>
>
...

> If a regression test for this is wanted, the pieces are mostly there --
> pg_stat_activity gives the io worker pids and the injection point above
> already hooks the reopen path -- but a portable assertion would probably
> need the worker's open-segment count exposed from inside the backend
> rather than read from /proc. We are happy to help with that if it is
> the direction people want.
>

I don't think we need a test for that at this stage, given that it is not an
observable behaviour. If we are to add a test case it would likely be along
the lines of enabling debug verbosity, and having a debug message
showing that the file was removed.

> POSSIBLE FIX DIRECTIONS (for discussion)
> ========================================
>
> 1. Have io workers call smgrcloseall() (or smgrreleaseall()) when they
> go idle, or on some bounded schedule -- the direct analogue of the
> bgwriter's checkpoint-time smgrcloseall(). Cheap and simple, at the
> cost of some reopens after an idle period.
>
> 2. Extend PROCSIGNAL_BARRIER_SMGRRELEASE, or add a narrower relation-
> scoped equivalent, so that relation drops also reach auxiliary
> processes. io workers already handle the barrier, so the delivery
> machinery exists; the question is the cost of a barrier per DROP.
>
> 3. Have io workers participate in shared invalidation. This is the most
> precise fix but the largest change, since it means giving an
> auxiliary process an invalidation slot and catchup handling.
>
> 4. Have smgr_aio_reopen() close the descriptor after the IO completes.
> Correct but presumably too expensive, as it reopens on every IO.
>
> We do not have a strong opinion on which is right; (1) looks like the
> smallest change that removes the unbounded behavior. We are happy to
> test any patch against both the reproducer and the production workload.
>

I implemented (3) the most precise, not sure if the largest.

I implemented this, the most precise, not too large
src/backend/storage/aio/method_worker.c | 31 +++++++++++++++++++++++++
1 file changed, 31 insertions(+)

REPRODUCER SCRIPT
>
I adapted your script to run faster 128kB shared buffers and a sub 1MB table
is good enough to show the issue.

Regards,
Alexandre

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message shihao zhong 2026-09-20 20:52:26 Re: BUG #19708: Hash Join becomes about 300x slower with higher work_mem
Previous Message Alexandre Felipe 2026-09-20 20:05:17 Re: BUG #19708: Hash Join becomes about 300x slower with higher work_mem