Skip site navigation (1) Skip section navigation (2)

Re: [HACKERS] Bgwriter behavior

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Simon Riggs <simon(at)2ndquadrant(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>,Gavin Sherry <swm(at)linuxworld(dot)com(dot)au>,PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>,pgsql-patches(at)postgresql(dot)org
Subject: Re: [HACKERS] Bgwriter behavior
Date: 2005-01-01 06:20:20
Message-ID: 200501010620.j016KKr13323@candle.pha.pa.us (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-patches
This change isn't going to make it for RC3, and it probably not
something we want to rush.

I think there are a few issues involved:

	o  everyone agrees the current meaning of bgwriter_percent is
	   useless (percent of dirty buffers)
	o  removal of bgwriter_percent will cause problems because
	   postgresql.conf is only installed via initdb, so beta users
	   will have to have some workaround so their existing
	   postgresql.conf files work.
	o  bgwriter_percent and bgwriter_maxpages are duplicate for a
	   given number of buffers and it isn't clear which one takes
	   precedence.
	o  8.1 might use these variables with different meanings,
	   causing slight upgrade confusion.
	o  Another idea is for bgwriter_percent to control how much of
	   the buffer is scanned.

Tom feels bgwriter_maxpages is good because it allows the user to
specify the I/O traffic, while bgwriter_percent as total pages (not just
dirty ones) is perhaps easier to set a default (I/O load varies based on
buffer cache size) and perhaps easier to understand.

I am not sure what to suggest at this point but whatever solution we use
should take the above issues into account.

---------------------------------------------------------------------------

Simon Riggs wrote:
> On Fri, 2004-12-31 at 01:14, Bruce Momjian wrote:
> > Simon Riggs wrote:
> > > On Mon, 2004-12-27 at 22:21, Bruce Momjian wrote:
> > > > Should we consider at least adjusting the meaning of bgwriter_percent?
> > > 
> > > Yes. As things stand, this is the only change that seems safe.
> > > 
> > > Here's a very short patch that implements this change within BufferSync
> > > in bufmgr.c 
> > > 
> > > - No algorithm changes
> > > - No error message changes
> > > - Only change is the call to StrategyDirtyBufferList is made using the
> > > maximum number of buffers that will be cleaned, rather than uselessly
> > > trawling through all of shared_buffers
> > > 
> > > This changes the meaning of bgwriter_percent from "percent of dirty
> > > buffers" to "percent of shared_buffers". The default settings of 1% of
> > > 1000 buffers gives up to 10 dirty block writes every 250ms
> > > 
> > > Benefit: allows performance tuning by increases options for setting
> > > bgwriter_delay which would otherwise have an ineffectually high minimum
> > > setting
> > > 
> > > Risk: low
> > > 
> > > 1-line doc patch to follow, if this is approved.
> > 
> > I am not objecting to the patch, but what value is there in having both
> > bgwriter_percent and bgwriter_maxpages?  Seems both are redundant and
> > that one would be enough.
> 
> In brief:
> i) for now: as little change as possible is good
> ii) the two parameters are OK
> iii) trying to decide an alternative takes time, which we do not have
> iv) what is presented here is simply a performance bug fix, not the best
> long term alternative...
> 
> I'd like to move quickly: if we do this (or an alternative), it has to
> be done soon and it would be easy to discuss this until we run out of
> time. Could we vote: in RC3, or not?
> 
> In more detail... 
> 
> The value of having both is:
> i) as little change as possible at this stage of RC - the main one
> ...which gives us stability
> ...and also avoids having to re-discuss what they *should* be
> 
> ii) Having two isn't that bad. bgwriter_percent auto adjusts the length
> of the to-be-cleaned-list, so it is roughly useful anywhere between 500
> and 10000 shared_buffers. That is IMHO slightly more useful than a hard
> definition set via bgwriter_maxpages, since that is likely to be set
> wrong anyway - but has some value as an outside limit on the number of
> pages. [You may wish to set shared_buffers > 10000 even on smaller
> servers, since many now have 2GB RAM and yet a relatively poor I/O
> subsystem. Having maxpages set separately allows the majority of people
> to set shared_buffers higher without swamping their I/O subsystems
> because they didn't know about the r8.0 bgwriter feature/parameters]
> 
> iii) changing the parameters might tempt us towards changing the
> algorithm, which is not a topic we have reached agreement on
> 
> iv) I see it as a goal to remove all of those parameters anyway, as well
> as explore some of the many options and ideas everybody has presented,
> so further change is likely at the next release whatever is done now.
> 
> The patch is as simple as I can make it and yet remove the unnecessary
> performance effect in the existing code. Thanks to Neil and others for
> showing that this was possible...I see this patch as a team effort.
> 
> I've already spoken against larger change and would do so again now: if
> we don't agree this change, then I would vote for no-change.... simply
> because this patch is minimal change. We *suspect* further change is
> beneficial but we have no evidence to support what that change should
> be, amongst the large range of possible solutions proposed.
> 
> -- 
> Best Regards, Simon Riggs
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>     (send "unregister YourEmailAddressHere" to majordomo(at)postgresql(dot)org)
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman(at)candle(dot)pha(dot)pa(dot)us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

In response to

Responses

pgsql-hackers by date

Next:From: Magnus HaganderDate: 2005-01-01 13:48:04
Subject: Re: UNICODE/UTF-8 on win32
Previous:From: Abhijit Menon-SenDate: 2005-01-01 06:02:03
Subject: Re: Copyright update

pgsql-patches by date

Next:From: Simon RiggsDate: 2005-01-01 15:43:55
Subject: Re: [HACKERS] Bgwriter behavior
Previous:From: Abhijit Menon-SenDate: 2005-01-01 06:02:03
Subject: Re: Copyright update

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group