Re: group locking: incomplete patch, just for discussion

On Wed, 2014-10-15 at 00:03 -0400, Robert Haas wrote:
> For parallelism, I think we need a concept of group locking. That is,
> suppose we have a user backend and N worker backends collaborating to
> execute some query. For the sake of argument, let's say it's a
> parallel CLUSTER, requiring a full table lock. We need all of the
> backends to be able to lock the table even though at least one of them
> holds AccessExclusiveLock. This suggests that the backends should all
> be members of a locking group, and that locks within the same locking
> group should be regarded as mutually non-conflicting.

Trying to catch up on this thread, please excuse me if these questions
are already covered.

You mention the possibility of undetected deadlocks, which is surely
unacceptable. But why not improve the deadlock detector? How bad are
_detected_ deadlocks? A lot of the concern was around catalog accesses,
but those lock requests would rarely wait anyway.

I also wonder if group locking is generally the wrong approach to
parallelism. Parallel scan/sort should work by assigning workers to
chunks of data, and then merging the results. In principle the workers
don't need to touch the same data at all, so why are we trying so hard
to get them to all take the same locks?

The reason, I assume, is that a full table is the lockable object, but
we are imagining the segment files as the chunks. But maybe there's a
way to address this more directly with an extra field in the lock tag,
and perhaps some catalog knowledge?

Regards,
Jeff Davis