Re: pg_background (and more parallelism infrastructure patches)

Hi,

On 2014-07-25 14:11:32 -0400, Robert Haas wrote:
> Attached is a contrib module that lets you launch arbitrary command in
> a background worker, and supporting infrastructure patches for core.

Cool.

I assume this 'fell out' of the work towards parallelism? Do you think
all of the patches (except the contrib one) are required for that or is
some, e.g. 3), only required to demonstrate the others?

> Patch 3 adds the ability for a backend to request that the protocol
> messages it would normally send to the frontend get redirected to a
> shm_mq. I did this by adding a couple of hook functions. The best
> design is definitely arguable here, so if you'd like to bikeshed, this
> is probably the patch to look at.

Uh. This doesn't sound particularly nice. Shouldn't this rather be
clearly layered by making reading/writing from the client a proper API
instead of adding hook functions here and there?

Also, you seem to have only touched receiving from the client, and not
sending back to the subprocess. Is that actually sufficient? I'd expect
that for this facility to be fully useful it'd have to be two way
communication. But perhaps I'm overestimating what it could be used for.

> This patch also adds a function to
> help you parse an ErrorResponse or NoticeResponse and re-throw the
> error or notice in the originating backend. Obviously, parallelism is
> going to need this kind of functionality, but I suspect a variety of
> other applications people may develop using background workers may
> want it too; and it's certainly important for pg_background itself.

I would have had use for it previously.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services