Re: SERIALIZABLE with parallel query

On Tue, Feb 21, 2017 at 12:55 AM, Thomas Munro
<thomas(dot)munro(at)enterprisedb(dot)com> wrote:
> On Thu, Feb 16, 2017 at 6:19 PM, Thomas Munro
> <thomas(dot)munro(at)enterprisedb(dot)com> wrote:
>> Specifically, DeleteChildTargetLocks() assumes it can walk
>> MySerializableXact->predicateLocks and throw away locks that are
>> covered by a new lock (ie throw away tuple locks because a covering
>> page lock has been acquired) without let or hindrance until it needs
>> to modify the locks themselves. That assumption doesn't hold up with
>> that last patch and will require a new kind of mutual exclusion. I
>> wonder if the solution is to introduce an LWLock into each
>> SERIALIZABLEXACT object, so DeleteChildTargetLocks() can prevent
>> workers from stepping on each others' toes during lock cleanup. An
>> alternative would be to start taking SerializablePredicateLockListLock
>> in exclusive rather than shared mode, but that seems unnecessarily
>> coarse.
>
> Here is a patch to do that, for discussion. It adds an LWLock to each
> SERIALIZABLEXACT, and acquires it after SerializablePredicateListLock
> and before any predicate lock partition lock. It doesn't bother with
> that if not in parallel mode, or in the cases where
> SerializablePredicateListLock is held exclusively. This prevents
> parallel query workers and leader from stepping on each others' toes
> when manipulating the predicate list.
>
> The case in CheckTargetForConflictsIn is theoretical for now since we
> don't support writing in parallel query yet. The case in
> CreatePredicateLock is reachable by running a simple parallel
> sequential scan. The case in DeleteChildTargetLocks is for when we've
> acquired a new predicate lock that covers finer grained locks which
> can be dropped; that is reachable the same way again. I don't think
> it's required in ReleaseOneSerializableXact since it was already
> called in several places with an sxact other than the caller's, and
> deals with finished transactions.

I don't think I know enough about the serializable code to review
this, or at least not quickly, but it seems very cool if it works.
Have you checked what effect it has on shared memory consumption?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company