Skip site navigation (1) Skip section navigation (2)

Re: [PATCHES] SRA Win32 sync() code

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: PostgreSQL-patches <pgsql-patches(at)postgresql(dot)org>,PostgreSQL Win32 port list <pgsql-hackers-win32(at)postgresql(dot)org>
Subject: Re: [PATCHES] SRA Win32 sync() code
Date: 2003-11-17 01:58:48
Message-ID: 200311170158.hAH1wmc06667@candle.pha.pa.us (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-hackers-win32pgsql-patches
Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom Lane wrote:
> >> Seriously though, if we can move the bulk of the writing work into
> >> background processes then I don't believe that there will be any
> >> significant penalty for regular backends.
> 
> > If the background writer starts using fsync(), we can have normal
> > backends that do a write() set a shared memory boolean.  We can then
> > test that boolean and do sync() only if other backends had to do their
> > own writes.
> 
> That seems like the worst of both worlds --- you still are depending on
> sync() for correctness.
> 
> Also, as long as backends only *seldom* do writes, making them fsync a
> write when they do make one will be less of an impact on overall system
> performance than having a sync() ensue shortly afterwards.  I think you
> are focusing too narrowly on the idea that backends shouldn't ever wait
> for writes, and failing to see the bigger picture.  What we need to
> optimize is overall system performance, not an arbitrary restriction
> that certain processes never wait for certain things.

OK, let me give you my logic and you can tell me where I am wrong.

First, how many backend can a single write process support if all the
backends are doing insert/update/deletes?  5?  10?  Let's assume 10. 
Second, once we change write to write/fsync, how much slower will that
be?  100x, 1000x?  Let's say 10x.

So, by my logic, if we have 100 backends all doing updates, we will need
10 * 100 or 1000 writer processes or threads to keep up with that load. 
That seems quite excessive to me from a context switching and process
overhead perspective.

Where am I wrong?

Also, if we go with the fsync only at checkpoint, we are doing fsync's
once every minute (at checkpoint time) rather than several times a
second potentially.

Do we know that having the background writer fsync a file that was
written by a backend cause all the data to fsync?  I think I could write
a program to test this by timing each of these tests:

	create an empty file
	open file
	time fsync
	close

	open file
	write 2mb into the file
	time fsync
	close

	open file
	write 2mb into the file
	close
	open file
	time fsync
	close
	
If I do the write via system(), I am doing it in a separate process so
the test should work.  Should I try this?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman(at)candle(dot)pha(dot)pa(dot)us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

In response to

Responses

pgsql-hackers by date

Next:From: Tom LaneDate: 2003-11-17 03:25:34
Subject: Re: [PATCHES] SRA Win32 sync() code
Previous:From: Marc G. FournierDate: 2003-11-17 01:23:20
Subject: Release now live ...

pgsql-patches by date

Next:From: Tom LaneDate: 2003-11-17 03:25:34
Subject: Re: [PATCHES] SRA Win32 sync() code
Previous:From: Bruce MomjianDate: 2003-11-17 00:27:38
Subject: Re: SIGPIPE handling

pgsql-hackers-win32 by date

Next:From: Tom LaneDate: 2003-11-17 03:25:34
Subject: Re: [PATCHES] SRA Win32 sync() code
Previous:From: Tom LaneDate: 2003-11-16 22:46:17
Subject: Re: [PATCHES] SRA Win32 sync() code

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group