Re: Configurable FP_LOCK_SLOTS_PER_BACKEND

Hi,

On 2023-08-07 13:05:32 -0400, Robert Haas wrote:
> I would also argue that the results are actually not that great,
> because once you get past 64 partitions you're right back where you
> started, or maybe worse off. To me, there's nothing magical about
> cases between 16 and 64 relations that makes them deserve special
> treatment - plenty of people are going to want to use hundreds of
> partitions, and even if you only use a few dozen, this isn't going to
> help as soon as you join two or three partitioned tables, and I
> suspect it hurts whenever it doesn't help.
>
> I think we need a design that scales better. I don't really know what
> that would look like, exactly, but your idea of a hybrid approach
> seems like it might be worth further consideration. We don't have to
> store an infinite number of fast-path locks in an array that we search
> linearly, and it might be better that if we moved to some other
> approach we could avoid some of the regression.

My gut feeling is that the state for fast path locks doesn't live in quite the
right place.

What if fast path locks entered PROCLOCK into the shared hashtable, just like
with normal locks, the first time a lock is acquired by a backend. Except that
we'd set a flag indicating the lock is a fastpath lock. When the lock is
released, neither the LOCALLOCK nor the PROCLOCK entry would be
removed. Instead, the LOCK/PROCLOCK would be modified to indicate that the
lock is not held anymore.

That itself wouldn't buy us much - we'd still need to do a lookup in the
shared hashtable. But, by the time we decide whether to use fast path locks,
we've already done a hash lookup in the LOCALLOCK hashtable. Because the
PROCLOCK entry would continue to exist, we can use LOCALLOCK->proclock to get
the PROCLOCK entry without a shared hash table lookup.

Acquiring a strong lock on a fastpath lock would basically entail modifying
all the relevant PROCLOCKs atomically to indicate that fast path locks aren't
possible anymore. Acquiring a fast path lock would just require atomically
modifying the PROCLOCK to indicate that the lock is held.

On a first blush, this sounds like it could end up being fairly clean and
generic?

Greetings,

Andres Freund