Skip site navigation (1) Skip section navigation (2)

Re: spinlock->pthread_mutex : real world results

From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Nils Goroll <slink(at)schokola(dot)de>
Cc: Jeff Janes <jeff(dot)janes(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Martijn van Oosterhout <kleptog(at)svana(dot)org>, Merlin Moncure <mmoncure(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: spinlock->pthread_mutex : real world results
Date: 2012-08-06 12:54:11
Message-ID: CA+TgmoZeMW5OZX1WzBXtieaTD0ySqJdauthoX_2NRZLc8_TxTw@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Sun, Aug 5, 2012 at 7:19 PM, Nils Goroll <slink(at)schokola(dot)de> wrote:
> meanwhile we're using the patch in production (again, this is 9.1.3) and
> after running it under full load for one week I believe it is pretty safe to
> say that replacing the spinlock code with pthread_mutexes on Linux (which
> basically are a futex wrapper) has solved the scalability issue and all
> stability/performance problems on this system are simply gone.
>
> While the improved pgbench run had already given a clear indication
> regarding the optimization potential, we can now be pretty certain that
> spinlock contention had really been the most significant root cause for the
> issues I had described in my early postings ("why roll-your-own s_lock? /
> improving scalability" / "experimental: replace s_lock spinlock code with
> pthread_mutex on linux").
>
> I am attaching annotated graphs showing the load averages and cpu statistics
> of the respective machine. Please note the fact that the highest spikes have
> been averaged out in these graphs. As I had mentioned before, with the
> original code in place we had seen saturation of 64 cores and load averages
> in excess of 300.
>
>
> I fully agree that improvements in more recent pgsql code to reduce the
> number of required locks or, even better, lockless data structures are the
> way to go, but for the remaining cases it should now have become apparent
> that favoring efficient mutex implementations is advantageous for large
> SMPs, where they exist (e.g. futexes on Linux).

Interesting data.  I guess the questions in my mind are:

1. How much we're paying for this in the uncontended case?

2. Should we be modifying our spinlock implementation on Linux to use
futexes rather than pulling pthreads into the mix?

Anyone have data on the first point, or opinions on the second one?

I certainly think there is some potential here in terms of preventing
the worst-case situation where the entire machine ends up spending a
major portion of its CPU time in s_lock.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

In response to

Responses

pgsql-hackers by date

Next:From: Amit KapilaDate: 2012-08-06 14:07:12
Subject: Re: WIP patch for LATERAL subqueries
Previous:From: Robert HaasDate: 2012-08-06 12:50:02
Subject: Re: Windows Streaming replication -- Windows 2008 servers

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group