Skip site navigation (1) Skip section navigation (2)

Re: Performance problems testing with Spamassassin

From: "Luke Lonergan" <LLonergan(at)greenplum(dot)com>
To: "Andrew McMillan" <andrew(at)catalyst(dot)net(dot)nz>,"Matthew Schumacher" <matt(dot)s(at)aptalaska(dot)net>
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: Performance problems testing with Spamassassin
Date: 2005-07-29 07:01:07
Message-ID: 3E37B936B592014B978C4415F90D662D0C4221@MI8NYCMAIL06.Mi8.com (view raw or flat)
Thread:
Lists: pgsql-performance
work_mem = 131072               # min 64, size in KB
shared_buffers = 16000          # min 16, at least max_connections*2, 8KB each
checkpoint_segments = 128       # in logfile segments, min 1, 16MB each
effective_cache_size = 750000   # typically 8KB each
fsync=false                     # turns forced synchronization on or off
 
------------------------------------------
On Bizgres (0_7_2) running on a 2GHz Opteron:
------------------------------------------
[llonergan(at)stinger4 bayesBenchmark]$ ./test.sh

real    0m38.348s
user    0m1.422s
sys     0m1.870s
 
------------------------------------------
On a 2.4GHz AMD64:
------------------------------------------
[llonergan(at)kite15 bayesBenchmark]$ ./test.sh

real    0m35.497s
user    0m2.250s
sys     0m0.470s
 
Now we turn fsync=true:
 
------------------------------------------
On a 2.4GHz AMD64:
------------------------------------------
[llonergan(at)kite15 bayesBenchmark]$ ./test.sh

real    2m7.368s
user    0m2.560s
sys     0m0.750s
 
I guess we see the real culprit here.  Anyone surprised it's the WAL?
 
- Luke

________________________________

From: pgsql-performance-owner(at)postgresql(dot)org on behalf of Andrew McMillan
Sent: Thu 7/28/2005 10:50 PM
To: Matthew Schumacher
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0



On Thu, 2005-07-28 at 16:13 -0800, Matthew Schumacher wrote:
>
> Ok, I finally got some test data together so that others can test
> without installing SA.
>
> The schema and test dataset is over at
> http://www.aptalaska.net/~matt.s/bayes/bayesBenchmark.tar.gz
>
> I have a pretty fast machine with a tuned postgres and it takes it about
> 2 minutes 30 seconds to load the test data.  Since the test data is the
> bayes information on 616 spam messages than comes out to be about 250ms
> per message.  While that is doable, it does add quite a bit of overhead
> to the email system.

On my laptop this takes:

real    1m33.758s
user    0m4.285s
sys     0m1.181s

One interesting effect is the data in bayes_vars has a huge number of
updates and needs vacuum _frequently_.  After the run a vacuum full
compacts it down from 461 pages to 1 page.

Regards,
                                        Andrew.

-------------------------------------------------------------------------
Andrew @ Catalyst .Net .NZ  Ltd,  PO Box 11-053, Manners St,  Wellington
WEB: http://catalyst.net.nz/            PHYS: Level 2, 150-154 Willis St
DDI: +64(4)803-2201      MOB: +64(272)DEBIAN      OFFICE: +64(4)499-2267
                      I don't do it for the money.
                    -- Donald Trump, Art of the Deal

-------------------------------------------------------------------------





Responses

pgsql-performance by date

Next:From: Bruce MomjianDate: 2005-07-29 12:37:00
Subject: Re: [PATCHES] COPY FROM performance improvements
Previous:From: Dennis BjorklundDate: 2005-07-29 06:48:48
Subject: Re: Performance problems testing with Spamassassin 3.1.0

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group