Re: Review: Extra Daemons / bgworker

Markus Wanner wrote:
> On 12/03/2012 03:28 PM, Alvaro Herrera wrote:

> >> Just like av_launcher does it now: set a flag in shared memory and
> >> signal the postmaster (PMSIGNAL_START_AUTOVAC_WORKER).
> >
> > I'm not sure how this works. What process is in charge of setting such
> > a flag?
>
> The only process that currently starts background workers ... ehm ...
> autovacuum workers is the autovacuum launcher. It uses the above
> Postmaster Signal in autovacuum.c:do_start_autovacuum_worker() to have
> the postmaster launch bg/autovac workers on demand.

Oh, I understand that. My question was more about what mechanism are
you envisioning for new bgworkers. What process would be in charge of
sending such signals? Would it be a persistent bgworker which would
decide to start up other bgworkers based on external conditions such as
timing? And how would postmaster know which bgworker to start -- would
it use the bgworker's name to find it in the registered workers list?

The trouble with the above rough sketch is how does the coordinating
bgworker communicate with postmaster. Autovacuum has a very, um,
peculiar mechanism to make this work: avlauncher sends a signal (which
has a hardcoded-in-core signal number) and postmaster knows to start a
generic avworker; previously avlauncher has set things up in shared
memory, and the generic avworker knows where to look to morph into the
specific bgworker required. So postmaster never really looks at shared
memory other than the signal number (which is the only use of shmem in
postmaster that's acceptable, because it's been carefully coded to be
robust). This doesn't work for generic modules because we don't have a
hardcoded signal number; if two modules wanted to start up generic
bgworkers, how would postmaster know which worker-main function to call?

Now, maybe we can make this work in some robust fashion ("robust"
meaning we don't put postmaster at risk, which is of utmost importance;
and this in turn means don't trust anything in shared memory.) I don't
say it's impossible; only that it needs some more thought and careful
design.

As posted, the feature is already useful and it'd be good to have it
committed soon so that others can experiment with whatever sample
bgworkers they see fit. That will give us more insight on other API
refinements we might need.

I'm going to disappear on paternity leave, most likely later this week,
or early next week; I would like to commit this patch before that. When
I'm back we can discuss other improvements. That will give us several
weeks before the end of the 9.3 development period to get these in. Of
course, other committers are welcome to improve the code in my absence.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services