Skip site navigation (1) Skip section navigation (2)

Re: lock contention on parallel COPY ?

From: Simon Riggs <simon(at)2ndQuadrant(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc>, Andrew Dunstan <andrew(at)dunslane(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: lock contention on parallel COPY ?
Date: 2008-09-26 17:56:36
Message-ID: 1222451796.4445.990.camel@ebony.2ndQuadrant (view raw or flat)
Thread:
Lists: pgsql-hackers
On Fri, 2008-09-26 at 12:38 -0400, Tom Lane wrote:
> Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc> writes:
> > heh no log archiving - I actually said that I'm now playing with 
> > --truncate-before-load which seems to cause a noticeable performance (as 
> > in IO generated) increase but I still see >130000 context switches/s and 
> > a profile that looks like:
> 
> > samples  %        symbol name
> > 55526    16.5614  LWLockAcquire
> > 29721     8.8647  DoCopy
> > 26581     7.9281  CopyReadLine
> > 25105     7.4879  LWLockRelease
> > 15743     4.6956  PinBuffer
> > 14725     4.3919  heap_formtuple
> 
> Still a lot of contention for something, then.  You might try turning on
> LWLOCK_STATS (this only requires recompiling storage/lmgr/lwlock.c) to
> get some evidence about what.

Probably loading a table with a generated PK or loading data in
ascending sequence, so its contending heavily for the rightmost edge of
the index.

We need to load data a block at a time and buffer the inserts into the
index also, so we don't need to lock/unlock per row.

-- 
 Simon Riggs           www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


In response to

Responses

pgsql-hackers by date

Next:From: Tom LaneDate: 2008-09-26 18:00:02
Subject: Re: lock contention on parallel COPY ?
Previous:From: Stefan KaltenbrunnerDate: 2008-09-26 17:53:03
Subject: Re: lock contention on parallel COPY ?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group