Skip site navigation (1) Skip section navigation (2)

Re: Final background writer cleanup for 8.3

From: Jan Wieck <JanWieck(at)Yahoo(dot)com>
To: Greg Smith <gsmith(at)gregsmith(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Final background writer cleanup for 8.3
Date: 2007-08-31 12:35:47
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-hackers
On 8/24/2007 1:17 AM, Greg Smith wrote:
> On Thu, 23 Aug 2007, Tom Lane wrote:
>> It is doubtless true in a lightly loaded system, but once the kernel is 
>> under any kind of memory pressure I think it's completely wrong.
> The fact that so many tests I've done or seen get maximum throughput in 
> terms of straight TPS with the background writer turned completely off is 
> why I stated that so explicitly.  I understand what you're saying in terms 
> of memory pressure, all I'm suggesting is that the empirical tests suggest 
> the current background writer even with moderate improvements doesn't 
> necessarily help when you get there.  If writes are blocking, whether the 
> background writer does them slightly ahead of time or whether the backend 
> does them itself doesn't seem to matter very much.  On a heavily loaded 
> system, your throughput is bottlenecked at the disk either way--and 
> therefore it's all the more important in those cases to never do a write 
> until you absolutely have to, lest it be wasted.

Have you used something that like a properly implemented TPC benchmark 
simulates users that go through cycles of think times instead of 
hammering SUT interactions at the maximum possible rate allowed by the 
network latency? And do your tests consider any completed transaction a 
good transaction, or are they like TPC benchmarks, which require the 
majority of transactions to complete in a certain maximum response time?

Those tests will show you that inflicting an IO storm at checkpoint time 
will delay processing enough to get a significant increase in the number 
of concurrent transactions by giving the "users" time enough to come out 
of their thinking time. That spike in active transactions increases 
pressure on CPU, memory and IO ... and eventually leads to the situation 
where users submit new transactions at a higher rate than you currently 
can commit ... which is where you enter the spiral of death.

Observing that very symptom during my TPC-W tests several years ago was 
what lead to developing the background writer in the first place. Can 
your tests demonstrate improvements for this kind of (typical web 
application) load profile?


# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck(at)Yahoo(dot)com #

In response to

pgsql-hackers by date

Next:From: Suresh_Date: 2007-08-31 12:37:37
Subject: Re: Performing antijoin in postgres
Previous:From: Albe LaurenzDate: 2007-08-31 12:34:09
Subject: Re: [GENERAL] Undetected corruption of table files

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group