Re: [HACKERS] fsync method checking

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: josh(at)agliodbs(dot)com
Cc: Kurt Roeckx <Q(at)ping(dot)be>, Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>, Mark Kirkwood <markir(at)paradise(dot)net(dot)nz>, pgsql-performance(at)postgresql(dot)org, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [HACKERS] fsync method checking
Date: 2004-03-18 21:00:54
Message-ID: 12261.1079643654@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-performance

Josh Berkus <josh(at)agliodbs(dot)com> writes:
> 1) This is an OSS project. Why not just recruit a bunch of people on
> PERFORMANCE and GENERAL to test the 4 different synch methods using real
> databases? No test like reality, I say ....

I agree --- that is likely to yield *far* more useful results than
any standalone test program, for the purpose of finding out what
wal_sync_method to use in real databases. However, there's a second
issue here: we would like to move sync/checkpoint responsibility into
the bgwriter, and that requires knowing whether it's valid to let one
process fsync on behalf of writes that were done by other processes.
That's got nothing to do with WAL sync performance. I think that it
would be sensible to make a test program that focuses on this one
specific question. (There has been some handwaving to the effect that
everybody knows this is safe on Unixen, but I question whether the
handwavers have seen the internals of HPUX or AIX for instance; and
besides we need to worry about Windows now.)

A third reason for having a simple test program is to confirm whether
your drives are syncing at all (cf. hdparm discussion).

> 2) Won't Jan's work on 7.5 memory and I/O management mean that we have to
> re-evaluate synching anyway?

So far nothing's been done that touches WAL writing. However, I am
thinking about making the bgwriter process take some of the load of
writing WAL buffers (right now it only writes data-file buffers).
And you're right, after that happens we will need to re-measure.
The open flags will probably become considerably more attractive than
they are now, if the bgwriter handles most non-commit writes of WAL.
(We might also think of letting the bgwriter use a different sync method
than the backends do.)

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2004-03-18 21:04:45 Re: [HACKERS] fsync method checking
Previous Message Bruce Momjian 2004-03-18 20:49:20 Re: [HACKERS] fsync method checking

Browse pgsql-performance by date

  From Date Subject
Next Message Tom Lane 2004-03-18 21:04:45 Re: [HACKERS] fsync method checking
Previous Message Bruce Momjian 2004-03-18 20:49:20 Re: [HACKERS] fsync method checking