Re: BBU Cache vs. spindles

On 10/20/2010 09:45 PM, Scott Marlowe wrote:
> On Wed, Oct 20, 2010 at 8:25 PM, Joshua D. Drake<jd(at)commandprompt(dot)com> wrote:
>
>> On Wed, 2010-10-20 at 22:13 -0400, Bruce Momjian wrote:
>>
>>> Ben Chobot wrote:
>>>
>>>> On Oct 7, 2010, at 4:38 PM, Steve Crawford wrote:
>>>>
>>>>
>>>>> I'm weighing options for a new server. In addition to PostgreSQL, this machine will handle some modest Samba and Rsync load.
>>>>>
>>>>> I will have enough RAM so the virtually all disk-read activity will be cached. The average PostgreSQL read activity will be modest - a mix of single-record and fairly large (reporting) result-sets. Writes will be modest as well but will come in brief (1-5 second) bursts of individual inserts. The rate of insert requests will hit 100-200/second for those brief bursts.
>>>>>
>>>>> So...
>>>>>
>>>>> Am I likely to be better off putting $$$ toward battery-backup on the RAID or toward adding a second RAID-set and splitting off the WAL traffic? Or something else?
>>>>>
>>>> A BBU is, what, $100 or so? Adding one seems a no-brainer to me.
>>>> Dedicated WAL spindles are nice and all, but they're still spinning
>>>> media. Raid card cache is waaaay faster, and while it's best at bursty
>>>> writes, it sounds like bursty writes are precisely what you have.
>>>>
>>> Totally agree!
>>>
>> BBU first, more spindles second.
>>
> Agreed. note that while you can get incredible burst performance from
> a battery backed cache, due to both caching and writing out of order,
> once the throughput begins to saturate at the speed of the disk array,
> the bbu cache is now only re-ordering really, as it will eventually
> fill up faster than the disks can take the writes, and you'll settle
> in at some percentage of your max tps you get for a short benchmark
> run. It's vitally important that once you put a BBU cache in place,
> you run a very long running transactional test (pgbench is a simple
> one to start with) that floods the io subsystem so you see what you're
> average throughput is with the WAL and data store getting flooded. I
> know on my system pgbench runs of a few minutes can be 3 or 4 times
> faster than runs that last for the better part of an hour.
>
>
Thanks for all the replies. This is what I suspected but since I can't
just buy one of everything to try, I wanted a sanity-check before
spending the $$$.

I am not too worried about saturating the controller cache as the
current much lower spec machine can handle the sustained load just fine
and the bursts are typically only 1-3 seconds long spaced a minute or
more apart.

Cheers,
Steve