Autovacuum multiworkers, try 6

From: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
To: Patches <pgsql-patches(at)postgresql(dot)org>
Subject: Autovacuum multiworkers, try 6
Date: 2007-04-10 21:22:26
Message-ID: 20070410212226.GH7786@alvh.no-ip.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-patches

Ok, here it is, after considerable rejiggering.

New in this patch is the handling of autovacuum PGPROC structs. They
are now reserved separately from regular backends'. As fallout it means
that it needs a separate code path in InitProcess to get it, and a
separate ProcKill. I had to add a separate spinlock to protect the
PGPROCs as well, since an LWLock doesn't cut it. (Note: this spinlock
is passed in the "save_backend_variables" stuff. I didn't see any
cleaner way to do this -- suggestions welcome.)

Also, I had to keep track of MaxConnections as set in postgresql.conf
separately MaxBackends, which is now
(MaxConnections+autovacuum_max_workers). A couple of assign hooks on
each take care of the INT_MAX/4 limit, which is applied to each
separately and also to the sum.

I put the PGPROCs and WorkerInfo structs as shared memory lists, similar
to the PGPROC struct in ProcGlobal. In the end this is somewhat cleaner
than the old coding of a fixed array of WorkerInfo. There is also a
shared memory pointer to a "starting" WorkerInfo, which is what is
picked up by the new worker process. This is more elegant than having
to cycle through all the WorkerInfos to see which one is "ready to be
used". It also makes a bit simpler the task of checking whether it's
not really starting due to some low-level problem (say, fork() failure).

There's also a shmem queue that keeps track of running workers. This
makes it simpler to check what table they are working on, as discussed.

Oh, Itagaki-san's patch to balance the delay limits has not been
integrated yet, but I'll work tomorrow on that unless he beats me to it
(and assuming nobody objects to the idea).

I appreciate review and testing.

I'm not taking the WIP label off this patch yet, but unless there are
serious objections I think this is close to final.

One last thing of note is that I haven't tried to compile it under
EXEC_BACKEND :-(

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Attachment Content-Type Size
autovacuum-multiworkers-6.patch text/x-diff 67.1 KB

Responses

Browse pgsql-patches by date

  From Date Subject
Next Message Alvaro Herrera 2007-04-10 21:36:15 Re: Autovacuum multiworkers, try 6
Previous Message Gurjeet Singh 2007-04-10 19:18:40 Re: [HACKERS] [Fwd: Index Advisor]