RE: [HACKERS] Moving relation extension locks out of heavyweight lock manager

--
Alex Ignatov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

-----Original Message-----
From: Alex Ignatov <a(dot)ignatov(at)postgrespro(dot)ru>
Sent: Monday, May 21, 2018 6:00 PM
To: 'Robert Haas' <robertmhaas(at)gmail(dot)com>; 'Andres Freund' <andres(at)anarazel(dot)de>
Cc: 'Masahiko Sawada' <sawada(dot)mshk(at)gmail(dot)com>; 'Michael Paquier' <michael(at)paquier(dot)xyz>; 'Mithun Cy' <mithun(dot)cy(at)enterprisedb(dot)com>; 'Tom Lane' <tgl(at)sss(dot)pgh(dot)pa(dot)us>; 'Thomas Munro' <thomas(dot)munro(at)enterprisedb(dot)com>; 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>; 'PostgreSQL-development' <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: [HACKERS] Moving relation extension locks out of heavyweight lock manager

-----Original Message-----
From: Robert Haas <robertmhaas(at)gmail(dot)com>
Sent: Thursday, April 26, 2018 10:25 PM
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>; Michael Paquier <michael(at)paquier(dot)xyz>; Mithun Cy <mithun(dot)cy(at)enterprisedb(dot)com>; Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>; Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>; Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>; PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

On Thu, Apr 26, 2018 at 3:10 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
>> I think the real question is whether the scenario is common enough to
>> worry about. In practice, you'd have to be extremely unlucky to be
>> doing many bulk loads at the same time that all happened to hash to
>> the same bucket.
>
> With a bunch of parallel bulkloads into partitioned tables that really
> doesn't seem that unlikely?

It increases the likelihood of collisions, but probably decreases the number of cases where the contention gets really bad.

For example, suppose each table has 100 partitions and you are bulk-loading 10 of them at a time. It's virtually certain that you will have some collisions, but the amount of contention within each bucket will remain fairly low because each backend spends only 1% of its time in the bucket corresponding to any given partition.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Hello!
I want to try to test this patch on 302(704 ht) core machine.

Patching on master (commit 81256cd05f0745353c6572362155b57250a0d2a0) is ok but got some error while compiling :

gistvacuum.c: In function ‘gistvacuumcleanup’:
gistvacuum.c:92:3: error: too many arguments to function ‘LockRelationForExtension’
LockRelationForExtension(rel, ExclusiveLock);
^
In file included from gistvacuum.c:21:0:
../../../../src/include/storage/extension_lock.h:30:13: note: declared here extern void LockRelationForExtension(Relation relation);
^
gistvacuum.c:95:3: error: too many arguments to function ‘UnlockRelationForExtension’
UnlockRelationForExtension(rel, ExclusiveLock);
^
In file included from gistvacuum.c:21:0:
../../../../src/include/storage/extension_lock.h:31:13: note: declared here extern void UnlockRelationForExtension(Relation relation);

Sorry, forgot to mention that patch version is extension-lock-v12.patch

--
Alex Ignatov
Postgres Professional: http://www.postgrespro.com The Russian Postgres Company