Skip site navigation (1) Skip section navigation (2)

Re: Moving more work outside WALInsertLock

From: Andres Freund <andres(at)anarazel(dot)de>
To: pgsql-hackers(at)postgresql(dot)org
Cc: Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
Subject: Re: Moving more work outside WALInsertLock
Date: 2011-12-15 14:32:02
Message-ID: 201112151532.02475.andres@anarazel.de (view raw or flat)
Thread:
Lists: pgsql-hackers
On Thursday, December 15, 2011 02:51:33 PM Heikki Linnakangas wrote:
> I've been looking at various ways to make WALInsertLock less of a
> bottleneck on multi-CPU servers. The key is going to be to separate the
> two things that are done while holding the WALInsertLock: a) allocating
> the required space in the WAL, and b) calculating the CRC of the record
> header and copying the data to the WAL page. a) needs to be serialized,
> but b) could be done in parallel.
> 
> I've been experimenting with different approaches to do that, but one
> thing is common among all of them: you need to know the total amount of
> WAL space needed for the record, including backup blocks, before you
> take the lock. So, here's a patch to move things around in XLogInsert()
> a bit, to accomplish that.
> 
> This patch doesn't seem to have any performance or scalability impact. I
> must admit I expected it to give a tiny gain in scalability by
> shortening the time WALInsertLock is held by a few instructions, but I
> can't measure any. But IMO it makes the code more readable, so this is
> worthwhile for that reason alone.
Thats great! I did (or at least tried) something similar when I was playing 
around with another crc32 implementation (which I plan to finish sometime). My 
changes where totally whacky but I got rather big improvements when changing 
the crc computation from incremental to one big swoop.
I started to hack up an api which buffered xlog data in statically sized buffer 
in each backend and only submitted that every now and then. Never got that to 
actually work correctly in more than the simplest cases though ;). In many 
cases were taking the wal insert lock way to often during a single insert... 
(you obviously know that...)

Andres


In response to

pgsql-hackers by date

Next:From: Robert HaasDate: 2011-12-15 14:36:25
Subject: Re: Command Triggers
Previous:From: Robert HaasDate: 2011-12-15 14:13:22
Subject: Re: Race condition in HEAD, possibly due to PGPROC splitup

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group