Skip site navigation (1) Skip section navigation (2)

Re: First set of OSDL Shared Mem scalability results, some

From: Curt Sampson <cjs(at)cynic(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Kevin Brown <kevin(at)sysexperts(dot)com>,pgsql-performance(at)postgresql(dot)org
Subject: Re: First set of OSDL Shared Mem scalability results, some
Date: 2004-10-23 07:33:40
Message-ID: Pine.NEB.4.58.0410231627240.1876@angelic-vtfw.cvpn.cynic.net (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-performance
On Sat, 9 Oct 2004, Tom Lane wrote:

> mmap provides msync which is comparable to fsync, but AFAICS it
> provides no way to prevent an in-memory change from reaching disk too
> soon.  This would mean that WAL entries would have to be written *and
> flushed* before we could make the data change at all, which would
> convert multiple updates of a single page into a series of write-and-
> wait-for-WAL-fsync steps.  Not good.  fsync'ing WAL once per transaction
> is bad enough, once per atomic action is intolerable.

Back when I was working out how to do this, I reckoned that you could
use mmap by keeping a write queue for each modified page. Reading,
you'd have to read the datum from the page and then check the write
queue for that page to see if that datum had been updated, using the
new value if it's there. Writing, you'd add the modified datum to the
write queue, but not apply the write queue to the page until you'd had
confirmation that the corresponding transaction log entry had been
written. So multiple writes are no big deal; they just all queue up in
the write queue, and at any time you can apply as much of the write
queue to the page itself as the current log entry will allow.

There are several different strategies available for mapping and
unmapping the pages, and in fact there might need to be several
available to get the best performance out of different systems. Most
OSes do not seem to be optimized for having thousands or tens of
thousands of small mappings (certainly NetBSD isn't), but I've never
done any performance tests to see what kind of strategies might work
well or not.

cjs
-- 
Curt Sampson  <cjs(at)cynic(dot)net>   +81 90 7737 2974   http://www.NetBSD.org
     Make up enjoying your city life...produced by BIC CAMERA

In response to

Responses

pgsql-performance by date

Next:From: Gaetano MendolaDate: 2004-10-23 10:31:32
Subject: Re: Insert performance, what should I expect?
Previous:From: Simon RiggsDate: 2004-10-22 22:01:05
Subject: Re: [PATCHES] ARC Memory Usage analysis

pgsql-hackers by date

Next:From: Peter EisentrautDate: 2004-10-23 13:40:35
Subject: to_char/to_number loses sign
Previous:From: Dennis BjorklundDate: 2004-10-23 06:09:05
Subject: Re: timestamp with time zone a la sql99

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group