Re: [GSOC][weekly report 9] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

In the last week, I tried these two ideas.

> -----Original Messages-----
> From: "Alvaro Herrera" <alvherre(at)2ndquadrant(dot)com>
> Sent Time: 2017-08-08 01:51:52 (Tuesday)
> * I wonder if a completely different approach to the finished xact
> maintenance problem would be helpful. For instance, in
> ReleasePredicateLocks we call ClearOldPredicateLocks()
> inconditionally, and there we grab the SerializableFinishedListLock
> lock inconditionally for the whole duration of that routine, but
> perhaps it would be better to skip the whole ClearOld stuff if the
> SerializableFinishedListLock is not available? Find out some way to
> ensure that the cleanup is always called later on, but maybe skipping
> it once in a while improves overall performance.
>

I implemented the idea by this way: using LWLockConditionalAcquire instead of LWLockAcquire.
If the lock is held by others, just return. It's OK because the routine is used to clear old predicate locks
but it doesn't matter who does the job.

But we both ignored one thing: this idea doesn't speedup the releasing operation. It just avoids some processes
waiting for the lock. If the function ClearOldPredicateLocks is the bottleneck, skipping doesn't help anymore.
I attached the result of evaluation. This idea ( conditional lock) has the worst performance.

>
> * the whole predicate.c stuff is written using SHM_QUEUE. I suppose
> SHM_QUEUE works just fine, but predicate.c was being written at about
> the same time (or a bit earlier) than the newer ilist.c interface was
> being created, which I think had more optimization work thrown in.
> Maybe it would be good for predicate.c to ditch use of SHM_QUEUE and
> use ilist.c interfaces instead? We could even think about being less
> strict about holding exclusive lock on SerializableFinished for the
> duration of ClearOldPredicateLocks, i.e. use only a share lock and
> only exchange for exclusive if a list modification is needed.
>

I used the double linked list in ilist.c but it didn't improve the performance.
I did a micro benchmark to compare the performance of SHM_QUEUE & ilist.c
and didn't find any difference. So the result is reasonable.

But there is a voice to get rid of SHM_QUEUE because it does not make sense
to have two same implementations. What's your opinion?

> --
> Álvaro Herrera https://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sincerely

Mengxing Liu