Re: Threading in BGWorkers (!)

On 06/22/20 23:38, Tom Lane wrote:
> bgworkers, or you could dispatch the threaded work to a non-Postgres-ish
> process as PL/Java does. The only advantage I can see of doing work in a
> process that's not at arm's-length is to have access to PG computational
> or IPC facilities, and none of that is likely to work safely in a threaded
> context.

You might be thinking of Dave Cramer's PL/JVM, which runs a JVM in another
process and does IPC to it.

PL/Java, by contrast, runs the JVM in-process and keeps both pieces.
It only lets one thread downcall into PostgreSQL.

On 06/23/20 00:46, Eric Ridge wrote:
> How can we start a dialog about this kind of situation? Nobody here is
> trying to make Postgres thread-safe, maybe only thread-friendly.

There are just a couple of things I've been wanting to suggest, based on
PL/Java experience.

1) It would be nice to be able to ereport from an arbitrary thread. There
is already support in core to forward messages from parallel workers:
the worker signals the lead process after adding a message to a shm_mq
referenced from its ParallelWorkerInfo struct. The signal handler
asynchronously sets ParallelMessagePending, which ProcessInterrupts
will check at some convenient point and ereport the message.

It seems like it would be no sweat for another thread in the same
process to add something to an mq (could be the same structure as
shm_mq but would not need to really be in shared memory) and do a
volatile write of ParallelMessagePending. The rest is already there.
Only missing ingredient would be a way for an extension to allocate
something like a ParallelWorkerInfo struct good for the life of the
backend (the current parallel worker infrastructure makes them all
go away at the completion of a parallel query).

2) It would be nice to be able to request service. If J Random thread
in PL/Java generates a bit of work requiring some PostgreSQL API,
at present that bit of work has to queue up until the next time a
call into PL/Java is occasioned by a query, which might be never.

It would be nice to be able to also asynchronously set some flag
like ExtensionServiceRequested, which could be checked as part of
CHECK_FOR_INTERRUPTS or even at more limited times, such as idle.
An extension could populate an ExtensionServiceInfo struct with
a service entry point an flag indicating that extension has work
pending.

Those are the two thread-friendlier ideas I have been thinking of.

Regards,
-Chap