Re: Performance issues with parallelism and LIMIT

On 11/18/25 16:07, David Geier wrote:
> Hi Tomas!
>
> On 18.11.2025 15:59, Tomas Vondra wrote:
>>>
>>> Some clarifications: I'm not inventing a new way to signal workers but
>>> I'm using the existing SendProcSignal() machinery to inform parallel
>>> workers to stop. I just added another signal PROCSIG_PARALLEL_STOP and
>>> the corresponding functions to handle it from ProcessInterrupts().
>>>
>>
>> Sure, but I still don't quite see the need to do all this.
>>
>>> What is "new" is how I'm stopping the parallel workers once they've
>>> received the stop signal: the challenge is that the workers need to
>>> actually jump out of whatever they are doing - even if they aren't
>>> producing any rows at this point; but e.g. are scanning a table
>>> somewhere deep down in ExecScan() / SeqNext().
>>>
>>> The only way I can see to make this work, without a huge patch that adds
>>> new code all over the place, is to instruct process termination from
>>> inside ProcessInterrupts(). I'm siglongjmp-ing out of the ExecutorRun()
>>> function so that all parallel worker cleanup code still runs as if the
>>> worker processed to completion. I've tried to end the process without
>>> but that caused all sorts of fallout (instrumentation not collected,
>>> postmaster thinking the process stopped unexpectedly, ...).
>>>
>>> Instead of siglongjmp-ing we could maybe call some parallel worker
>>> shutdown function but that would require access to the parallel worker
>>> state variables, which are currently not globally accessible.
>>>
>>
>> But why? The leader and workers already share state - the parallel scan
>> state (for the parallel-aware scan on the "driving" table). Why couldn't
>> the leader set a flag in the scan, and force it to end in workers? Which
>> AFAICS should lead to workers terminating shortly after that.
>>
>> All the code / handling is already in place. It will need a bit of new
>> code in the parallel scans, but but not much I think.
>>
>
> But this would only work for the SeqScan case, wouldn't it? The parallel
> worker might equally well be executing other code which doesn't produce
> tuples, such as parallel index scan, a big sort, building a hash table, etc.
>
> I thought this is not a viable solution because it would need changes in
> all these places.

It'd need code in the parallel-aware scans, i.e. seqscan, bitmap, index.
I don't think you'd need code in other plans, because all parallel plans
have one "driving" table.

Maybe that's not enough, not sure. If we want to terminate "immediately"
(and not when getting the next tuple from a scan on the driving table),
we'd need a different solution.

regards

--
Tomas Vondra