bg worker: patch 2 of 6 - job cache

From: Markus Wanner <markus(at)bluegap(dot)ch>
To: PostgreSQL-development Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: bg worker: patch 2 of 6 - job cache
Date: 2010-07-13 14:31:07
Message-ID: 4C3C78AB.1080707@bluegap.ch
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

This is the major refactoring patch, which turns the background workers
into more permanent work-horses. After having processed a job, they now
stay connected to their database and wait for more jobs from the
coordinator.

On the coordinator side, a job cache got added, so it is able to queue
requests for jobs on databases which don't currently have an idle
background worker available. The existing autovacuum functionality got
separated from the background worker infrastructure.

The coordinator uses a transaction to read pg_database and start
background workers as appropriate. However, autovacuum still uses
pg_stat, which needs to get initialized after startup. The coordinator
triggers an initial VACUUM on the template database (template1), which
populates the required statistics to kick off autovacuum.

Note that none of the statistics functions differenciate between
autovacuum workers and other background job types, which might or might
not be what we want.

Another open issue is the starvation problem: jobs for a database, which
doesn't currently have any connected background workers might starve, if
the coordinator isn't allowed to fork any new worker (and all others are
idle on other databases).

Attachment Content-Type Size
step2-job_cache.diff text/x-diff 61.0 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Markus Wanner 2010-07-13 14:31:13 bg worker: patch 3 of 6 - sockets
Previous Message Markus Wanner 2010-07-13 14:31:00 bg worker: patch 1 of 6 - permanent process