| From: | Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com> |
|---|---|
| To: | shihao zhong <zhong950419(at)gmail(dot)com> |
| Cc: | Alexandre Felipe <o(dot)alexandre(dot)felipe(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: aio: worker: Free SMGR objects when idle |
| Date: | 2026-09-23 11:35:01 |
| Message-ID: | CAN55FZ2JTHmB+xRQQd8LPE0vO0c=95yP8BHHJmZBe93M1u_SLA@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
Thank you for looking into this all!
On Tue, 22 Sept 2026 at 03:12, shihao zhong <zhong950419(at)gmail(dot)com> wrote:
>
> Right, the per IO check puts the spinlock in the hot path.
>
> The barrier also only releases, it does not destroy, so it does not
> fix what Nazir reported. The hash table in the IO worker keeps
> growing either way. Nothing pins those entries in an IO worker, and
> smgrdestroyall only zaps unpinned ones, so destroying them there
> should be safe.
Correct.
> That suggests a version with no lock at all. Keep the cleanup where
> Nazir put it, at a safe point in the worker loop, but trigger it on a
> local condition, the number of unpinned entries being over a cap,
> rather than on a checkpoint generation. No shared state, and it also
> covers a worker that never goes idle.
We need to make a function call to hash_get_num_entries() for each IO
to check number of unpinned entries, which doesn't sound good to me.
This is especially true for the partitioned case.
For now, I used the number of IOs (capped at 1024) since the last smgr
cleanup. This solution removes CheckpointerShmem->ckpt_lck contention
and worker goes idle problem.
--
Regards,
Nazir Bilal Yavuz
Microsoft
| Attachment | Content-Type | Size |
|---|---|---|
| v2-0001-aio-worker-Periodically-free-SMGR-objects.patch | text/x-patch | 2.4 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | shihao zhong | 2026-09-23 12:08:31 | Re: REPACK (CONCURRENTLY) can silently lose updates when the toast table is rewritten |
| Previous Message | Vik Fearing | 2026-09-23 11:34:35 | Re: [PATCH] Add ALTER SYSTEM RELOAD |