Re: Re[2]: [HACKERS] Connect from background worker thread to database

From: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
To: Alexey Vasiliev <leopard_ne(at)inbox(dot)ru>
Cc: PostgreSQL mailing lists <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Re[2]: [HACKERS] Connect from background worker thread to database
Date: 2013-11-25 04:31:15
Message-ID: CAB7nPqTOeyvUNOaYgKopMme4pXSfxiFm8knb7Gs23sr=5FQoHw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Nov 25, 2013 at 1:02 PM, Alexey Vasiliev <leopard_ne(at)inbox(dot)ru> wrote:
> Thanks, I just try this. This work if I working with database in loop inside
> bgw_main function. But if I create threads inside bgw_main and try to
> connect to database - I have errors "stack depth limit exceeded" . The code:
>
> https://github.com/le0pard/pg_web/blob/master/src/pg_web.c#L195 - connect to
> database
> https://github.com/le0pard/pg_web/blob/master/src/pg_web.c#L100-L131 -
> "http_event_handler" function execute in threads, because it handle http
> requests. And this code not work. BTW, I need connect to database depend
> from request url, so execute BackgroundWorkerInitializeConnection at the
> beginning not enough.
There is a design problem with your application when trying to create
threads with mg_start in order to grab events. Note that I am not
familiar with mongoose, but with bgworkers you cannot simply create
new threads that would be able to connect to server concurrently. A
model that would be more suited with bgworkers would be something
like:
- Handle event messages by for example opening a port or monitoring
the events on a single bgworker launched by bgw_main, with for example
a message queue model (possible with 9.3). Connection to database
would be done with a single connection, managed within the loop of
bgw_main.
- Create a new bgworker dynamically each time a new event comes in
(possible only with 9.4~). The top work would be done by a central
bgworker initialized with server, which would treat events and kick
new slave workers when necessary.

Regards,
--
Michael

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Noah Misch 2013-11-25 04:49:16 Re: UNION ALL on partitioned tables won't use indices.
Previous Message Alexey Vasiliev 2013-11-25 04:02:18 Re[2]: [HACKERS] Re[2]: [HACKERS] Connect from background worker thread to database