Re: bg worker: overview

Markus Wanner <markus(at)bluegap(dot)ch> writes:
> Daemon code? That sounds like it could be an addition to the
> coordinator, which I'm somewhat hesitant to extend, as it's a pretty
> critical process (especially for Postgres-R).
[...]
> However, note that the coordinator is designed to be just a message
> passing or routing process, which should not do any kind of time
> consuming processing. It must *coordinate* things (well, jobs) and react
> promptly. Nothing else.

Yeah, I guess user daemons would have to be workers, not plugins you
want to load into the coordinator.

> On the other side, the background workers have a connection to exactly
> one database. They are supposed to do work on that database.

Is that because of the way backends are started, and to avoid having to
fork new ones too often?

> The background workers can easily load external libraries - just as a
> normal backend can with LOAD. That would also provide better
> encapsulation (i.e. an error would only tear down that backend, not the
> coordinator). You'd certainly have to communicate between the
> coordinator and the background worker. I'm not sure how match that fits
> your use case.

Pretty well I think.

> The thread on -performance is talking quite a bit about connection
> pooling. The only way I can imagine some sort of connection pooling to
> be implemented on top of bgworkers would be to let the coordinator
> listen on an additional port and pass on all requests to the bgworkers
> as jobs (using imessages). And of course send back the responses to the
> client. I'm not sure how that overhead compares to using pgpool or
> pgbouncer. Those are also separate processes through which all of your
> data must flow. They use plain system sockets, imessages use signals and
> shared memory.

Yeah. The connection pool is better outside of code. Let's think PGQ and
a internal task scheduler first, if we think at any generalisation.

Regards,
--
dim