Re: [HACKERS] Performance while loading data and indexing

From: Greg Copeland <greg(at)CopelandConsulting(dot)Net>
To: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
Cc: shridhar_daithankar(at)persistent(dot)co(dot)in, PostgresSQL Hackers Mailing List <pgsql-hackers(at)postgresql(dot)org>, PostgresSQL General Mailing List <pgsql-general(at)postgresql(dot)org>
Subject: Re: [HACKERS] Performance while loading data and indexing
Date: 2002-09-26 17:44:22
Message-ID: 1033062262.23475.16.camel@mouse.copelandconsulting.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers pgsql-performance

On Thu, 2002-09-26 at 11:41, Bruce Momjian wrote:
> Shridhar Daithankar wrote:
> > I might have found the bottleneck, although by accident. Mysql was running out
> > of space while creating index. So my friend shut down mysql and tried to move
> > things by hand to create links. He noticed that even things like cp were
> > terribly slow and it hit us.. May be the culprit is the file system. Ext3 in
> > this case.
>
> I just added a file system and multi-cpu section to my performance
> tuning paper:
>
> http://www.ca.postgresql.org/docs/momjian/hw_performance/
>
> The paper does recommend ext3, but the differences between file systems
> are very small. If you are seeing 'cp' as slow, I wonder if it may be
> something more general, like poorly tuned hardware or something. You can
> use 'dd' to throw some data around the file system and see if that is
> showing slowness; compare those numbers to another machine that has
> different hardware/OS.
>
> Also, though ext3 is slower, turning fsync off should make ext3 function
> similar to ext2. That would be an interesting test if you suspect ext3.

I'm curious as to why you recommended ext3 versus some other (JFS,
XFS). Do you have tests which validate that recommendation or was it a
simple matter of getting the warm fuzzies from familiarity?

Greg

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message scott.marlowe 2002-09-26 18:41:55 Re: [HACKERS] Performance while loading data and indexing
Previous Message Greg Copeland 2002-09-26 17:36:57 Re: [HACKERS] Performance while loading data and indexing

Browse pgsql-hackers by date

  From Date Subject
Next Message scott.marlowe 2002-09-26 18:41:55 Re: [HACKERS] Performance while loading data and indexing
Previous Message Marc G. Fournier 2002-09-26 17:39:06 ignore

Browse pgsql-performance by date

  From Date Subject
Next Message scott.marlowe 2002-09-26 18:41:55 Re: [HACKERS] Performance while loading data and indexing
Previous Message Greg Copeland 2002-09-26 17:36:57 Re: [HACKERS] Performance while loading data and indexing