Re: Threading in BGWorkers (!)

Hi,

On 2020-06-23 09:19:36 -0400, Chapman Flack wrote:
> 1) It would be nice to be able to ereport from an arbitrary thread. There
> is already support in core to forward messages from parallel workers:
> the worker signals the lead process after adding a message to a shm_mq
> referenced from its ParallelWorkerInfo struct. The signal handler
> asynchronously sets ParallelMessagePending, which ProcessInterrupts
> will check at some convenient point and ereport the message.
>
> It seems like it would be no sweat for another thread in the same
> process to add something to an mq (could be the same structure as
> shm_mq but would not need to really be in shared memory) and do a
> volatile write of ParallelMessagePending. The rest is already there.
> Only missing ingredient would be a way for an extension to allocate
> something like a ParallelWorkerInfo struct good for the life of the
> backend (the current parallel worker infrastructure makes them all
> go away at the completion of a parallel query).

I think that's way harder than what you make it sound here. The locking
for shm_mq doesn't really work inside a process. In contrast to the
single threaded case something like a volatile write to
ParallelMessagePending doesn't guarantee much, because there's no
guaranteed memory ordering between threads. And more.

I'm very doubtful this is a good direction to go in. Kinda maybe
somewhat partially converting tiny parts of the backend code into
threadsafe code will leave us with some baroque code.

Greetings,

Andres Freund