Re: Scaling XLog insertion (was Re: Moving more work outside WALInsertLock)

On 07.03.2012 17:28, Tom Lane wrote:
> Simon Riggs<simon(at)2ndQuadrant(dot)com> writes:
>> On Wed, Mar 7, 2012 at 3:04 PM, Tom Lane<tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Alvaro Herrera<alvherre(at)commandprompt(dot)com> writes:
>>>> So they are undoubtely rare. Not sure if as rare as Higgs bosons.
>
>>> Even if they're rare, having a major performance hiccup when one happens
>>> is not a side-effect I want to see from a patch whose only reason to
>>> exist is better performance.
>
>> I agree the effect you point out can exist, I just don't want to slow
>> down the main case as a result.
>
> I don't see any reason to think that what I suggested would slow things
> down, especially not if the code were set up to fall through quickly in
> the typical case where no page boundary is crossed. Integer division is
> not slow on any machine made in the last 15 years or so.

Agreed. I wasn't worried about the looping with extra-large records, but
might as well not do it.

Here's an updated patch. It now only loops once per segment that a
record crosses. Plus a lot of other small cleanup.

I've been doing some performance testing with this, using a simple C
function that just inserts a dummy WAL record of given size. I'm not
totally satisfied. Although the patch helps with scalability at 3-4
concurrent backends doing WAL insertions, it seems to slow down the
single-client case with small WAL records by about 5-10%. This is what
Robert also saw with an earlier version of the patch
(http://archives.postgresql.org/pgsql-hackers/2011-12/msg01223.php). I
tested this with the data directory on a RAM drive, unfortunately I
don't have a server with a hard drive that can sustain the high
insertion rate. I'll post more detailed results, once I've refined the
tests a bit.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com