| From: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> |
|---|---|
| To: | Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> |
| Cc: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: Parallel vacuum: workers together with leader could overrun cost budget |
| Date: | 2026-10-20 21:01:20 |
| Message-ID: | CAD21AoCxu0N-AN47y_uYAjDKYBMiQ0z6a1BOuTYb6gBHJke_QA@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Tue, Oct 20, 2026 at 9:38 AM Bharath Rupireddy
<bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
>
> Hi,
>
> On Fri, Oct 16, 2026 at 10:46 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> >
> > Thank you for the report and the patch. I agree with your analysis.
> > Currently each process sleeps in proportion to the cost it incurred by
> > itself, so the leader and its workers together consume N times the
> > budget. I think the cost limit should work like a rate limit of a
> > download manager for example: using 5 connections makes the download
> > after, but if we set the limit to 1MB/s, we expect 1MB/s in total ,
> > not 1MB/s * 5 = 5MB/s.
>
> Nice analogy!
>
> > > Although this exists for parallel vacuum too, back-patching it to
> > > PG19, where parallel autovacuum was introduced, is fine, since
> > > parallel autovacuum runs with the default cost delay and cost limit.
> >
> > Agree to backpatch it to v19. Autovaucum distributes the cost limit
> > among the running autovacuum workers so that the sum doesn't exceed
> > autovacuum_vacuum_cost_limit, but parallel vacuum workers are not
> > counted there.
> >
> > I've reviewed the patch and have one comment:
> >
> > + /*
> > + * Compute sleep time based on the local cost balance and the number
> > + * of active workers.
> > + */
> > + msec = vacuum_cost_delay * VacuumCostBalanceLocal * nworkers /
> > + vacuum_cost_limit;
> >
> > msec can be up to nworkers * vacuum_cost_delay, but
> > vacuum_delay_point() caps the sleep time at 4 * vacuum_cost_delay.
> > Therefore, when more than 4 parallel workers are active, the sleep
> > time is truncated and we still overrun the budget. IIUC the cap was
> > introduced to avoid uselessly long delays when the cost accumulates
> > far beyond the limit.
>
> Ah, you are right. The long delays are avoided when the cost
> accumulates far higher than the specified limit. E.g., while holding
> critical locks or when interrupts are pending, the cost-based delays
> are not applied, so the cost keeps accumulating until the locks are
> released or the interrupts are processed.
>
> > I think a simple fix is to sleep up to 4 *
> > nworkers * vacuum_cost_delay at once. I think that's acceptable since
> > the sleep time is still proportional to the cost the process actually
> > incurred.
>
> Agreed. The non-parallel vacuum caps the sleep time whenever the
> accumulated cost balance crosses 4 times the cost limit. We should
> just do the similar capping for each worker while not letting all of
> them overrun the cost budget.
>
> Please find the attached v2 patch with the above changes and some doc
> adjustments.
Thank you for updating the patch! The patch looks good to me. It's
nice to update the documentation part about (vacuum_cost_delay * 4 *
nworkers) limit.
I think this patch should be backpatched to all supported branches
because the VACUUM command with the PARALLEL option should honor the
cost-based delay although it's disabled by default. What do you think?
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
| From | Date | Subject | |
|---|---|---|---|
| Next Message | surya poondla | 2026-10-20 21:24:56 | Re: tablecmds: Deadlock caused by Attach Partition |
| Previous Message | Masahiko Sawada | 2026-10-20 20:52:50 | Re: Reject COPY TO of unlogged relations during recovery |