Re: hung backends stuck in spinlock heavy endless loop

Hi,

On 2015-01-13 16:29:51 -0600, Merlin Moncure wrote:
> On my workstation today (running vanilla 9.4.0) I was testing some new
> code that does aggressive parallel loading to a couple of tables. It
> ran ok several dozen times and froze up with no external trigger.
> There were at most 8 active backends that were stuck (the loader is
> threaded to a cap) -- each query typically resolves in a few seconds
> but they were hung for 30 minutes+.

Interesting.

> Had to do restart immediate as
> backends were not responding to cancel...but I snapped a 'perf top'
> before I did so. The results were interesting so I'm posting them
> here. So far I have not been able to reproduce...FYI

Can you compile postgres with -fno-omit-frame-pointer? Then, when this
happens the next time, you can take a perf record -g, which will tell us
which lock the contention is at.

> 61.03% postgres [.] s_lock
> 13.56% postgres [.] LWLockRelease
> 10.11% postgres [.] LWLockAcquire

That profile looks like it might end up being helped by the lwlock
and/or freelist changes in 9.5.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services