Re: Threading in BGWorkers (!)

On 06/23/20 22:13, Tom Lane wrote:
> 1. elog.c is in itself not-thread-safe, because it uses a static data
> structure to track the message being built.
>
> 2. It uses palloc, another large pile of not-thread-safe infrastructure.
...

I'm sure now that I shouldn't have mentioned (1) - muddied waters. The idea
in my head had been to make what the PG code sees as close to the parallel-
message case as possible: "here is an error structure that your elog code
did not build, in a region of memory your palloc code did not manage." But
leave that aside, because a way to request a service callback would clearly
allow the regular elog and the regular palloc to do their regular thing
on the regular thread, and be the more general and desirable idea anyway.

> I'm not sure whether the other idea
>
>>> It would be nice to be able to also asynchronously set some flag
>>> like ExtensionServiceRequested, which could be checked as part of
>>> CHECK_FOR_INTERRUPTS or even at more limited times, such as idle.
>
> is much easier. In the barest terms, we already have things like that
> (such as NOTIFY interrupts), so it doesn't sound hard at first. The
> problem is to figure out whether action X that you wish to do is safe
> to do at CHECK_FOR_INTERRUPTS call site Y. The answer is certainly not
> always "yes", but how would we build an infrastructure for deciding?
> (NOTIFY largely punts on this, by decreeing that it won't do anything
> till we reach an idle state. That's surely not adequate for a lot
> of likely actions X.)

I think it could be adequate for a lot of them. (I even said "more
limited times, such as idle" up there.) In PL/Java's case,
there clearly aren't people running code now that functionally depends
on this ability, because it wouldn't work. Even if the JVM uses multiple
threads to accomplish something, if it is something the Java function
result depends on, it obviously has to happen before the function returns,
while PL/Java has the main thread and can just serialize the work onto it.

The likeliest cases where something might want to happen after the
function has returned are resource releases, which can sometimes
be discovered by the garbage collector a little after the fact, and if
the Java resource that's being collected is the dual of some palloc'd
or reference-counted PostgreSQL object, it would be nice to not have to
enqueue that cleanup for the next time some query calls into PL/Java.
Even an "only in an idle state" rule would be an improvement over
"who knows when and maybe never".

Regards,
-Chap