Skip site navigation (1) Skip section navigation (2)

Re: bg worker: overview

From: Dimitri Fontaine <dfontaine(at)hi-media(dot)com>
To: Markus Wanner <markus(at)bluegap(dot)ch>
Cc: PostgreSQL-development Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: bg worker: overview
Date: 2010-07-23 19:45:36
Message-ID: m2y6d2q8wv.fsf@hi-media.com (view raw or flat)
Thread:
Lists: pgsql-hackers
Markus Wanner <markus(at)bluegap(dot)ch> writes:
> Daemon code? That sounds like it could be an addition to the
> coordinator, which I'm somewhat hesitant to extend, as it's a pretty
> critical process (especially for Postgres-R).
[...]
> However, note that the coordinator is designed to be just a message
> passing or routing process, which should not do any kind of time
> consuming processing. It must *coordinate* things (well, jobs) and react
> promptly. Nothing else.

Yeah, I guess user daemons would have to be workers, not plugins you
want to load into the coordinator.

> On the other side, the background workers have a connection to exactly
> one database. They are supposed to do work on that database.

Is that because of the way backends are started, and to avoid having to
fork new ones too often?

> The background workers can easily load external libraries - just as a
> normal backend can with LOAD. That would also provide better
> encapsulation (i.e. an error would only tear down that backend, not the
> coordinator). You'd certainly have to communicate between the
> coordinator and the background worker. I'm not sure how match that fits
> your use case.

Pretty well I think.

> The thread on -performance is talking quite a bit about connection
> pooling. The only way I can imagine some sort of connection pooling to
> be implemented on top of bgworkers would be to let the coordinator
> listen on an additional port and pass on all requests to the bgworkers
> as jobs (using imessages). And of course send back the responses to the
> client. I'm not sure how that overhead compares to using pgpool or
> pgbouncer. Those are also separate processes through which all of your
> data must flow. They use plain system sockets, imessages use signals and
> shared memory.

Yeah. The connection pool is better outside of code. Let's think PGQ and
a internal task scheduler first, if we think at any generalisation.

Regards,
-- 
dim

In response to

Responses

pgsql-hackers by date

Next:From: Andres FreundDate: 2010-07-23 19:46:15
Subject: Re: Rewrite, normal execution vs. EXPLAIN ANALYZE
Previous:From: Robert HaasDate: 2010-07-23 19:32:52
Subject: Re: Rewrite, normal execution vs. EXPLAIN ANALYZE

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group