Re: autovacuum maintenance_work_mem

From: Josh Berkus <josh(at)agliodbs(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: autovacuum maintenance_work_mem
Date: 2010-11-16 18:52:14
Message-ID: 4CE2D2DE.9010801@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


> I think the difficulty is figuring out what to get the existing
> workers to give us some memory when a new one comes along. You want
> the first worker to potentially use ALL the memory... until worker #2
> arrives.

Yeah, doing this would mean that you couldn't give worker #1 all the
memory, because on most OSes it can't release the memory even if it
wants to.

The unintelligent way to do this is to do it by halves, which is what I
think Oracle 8 did for sort memory, and what the old time-sharing
systems used to do. That is, the first worker can use up to 50%. The
second worker gets up to 25%, or what's left over from the first worker
+ 25%, whichever is greater, up to 50%. The third worker gets similar,
up to max_workers. This kind of a system actually works pretty well,
*except* that it causes underallocation in the case (the most common
one, I might add) where only one worker is needed.

The intelligent way would be for Postgres to track the maximum
concurrent workers which have been needed in the past, and allocate
(again, by halves) based on that number.

Either of these systems would require some new registers in shared
memory to track such things.

Relevant to this is the question: *when* does vacuum do its memory
allocation? Is memory allocation reasonably front-loaded, or does
vacuum keep grabbing more RAM until it's done?

--
-- Josh Berkus
PostgreSQL Experts Inc.
http://www.pgexperts.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2010-11-16 18:58:13 Re: unlogged tables
Previous Message Heikki Linnakangas 2010-11-16 18:50:49 Re: GiST insert algorithm rewrite