Re: bg worker: patch 1 of 6 - permanent process

Hi,

I'm glad discussion on this begins.

On 09/14/2010 07:55 PM, Tom Lane wrote:
> So there is a minimum of one avworker per database?

Nope, you can set that to 0. You don't *need* to keep idle workers around.

> That's a guaranteed
> nonstarter. There are many people with thousands of databases, but no
> need for thousands of avworkers.

Well, yeah, bgworkers are primarily designed to be used for Postgres-R,
where you easily get more background workers than normal backends. And
having idle backends around waiting for a next job is certainly
preferable over having to re-connect every time.

I've advertised the bgworker infrastructure for use for parallel
querying as well. Again, that use case easily leads to having more
background workers than normal backends. And you don't want to wait for
them all to re-connect for every query they need to help with.

> I'm also pretty unclear why you speak of min and max numbers of workers
> when the proposal (AIUI) is to have the workers there always, rather
> than have them come and go.

This step 1 of the bgworker set of patches turns the av*launcher*
(coordinator) into a permanent process (even if autovacuum is off).

The background workers can still come and go. However, they don't
necessarily *need* to terminate after having done their job. The
coordinator controls them and requests new workers or commands idle ones
to terminate *as required*.

I don't think there's that much different to the current implementation.
Setting both, the min and max number of idle bgworkers to 0 should in
fact give you the exact same behavior as we currently have: namely to
terminate each av/bgworker after its job is done, never having idle
workers around. Which might or might not be the optimal configuration
for users with lots of databases, that's hard to predict. And it depends
a lot on the load distribution over the databases and on how clever the
coordinator manages the bgworkers.

Regards

Markus Wanner