Re: [HACKERS] Performance while loading data and indexing

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Greg Copeland <greg(at)copelandconsulting(dot)net>
Cc: shridhar_daithankar(at)persistent(dot)co(dot)in, PostgresSQL Hackers Mailing List <pgsql-hackers(at)postgresql(dot)org>, PostgresSQL General Mailing List <pgsql-general(at)postgresql(dot)org>
Subject: Re: [HACKERS] Performance while loading data and indexing
Date: 2002-09-26 20:00:48
Message-ID: 200209262000.g8QK0mG10553@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers pgsql-performance

Greg Copeland wrote:
> > The paper does recommend ext3, but the differences between file systems
> > are very small. If you are seeing 'cp' as slow, I wonder if it may be
> > something more general, like poorly tuned hardware or something. You can
> > use 'dd' to throw some data around the file system and see if that is
> > showing slowness; compare those numbers to another machine that has
> > different hardware/OS.
> >
> > Also, though ext3 is slower, turning fsync off should make ext3 function
> > similar to ext2. That would be an interesting test if you suspect ext3.
>
> I'm curious as to why you recommended ext3 versus some other (JFS,
> XFS). Do you have tests which validate that recommendation or was it a
> simple matter of getting the warm fuzzies from familiarity?

I used the attached email as a reference. I just changed the wording to
be:

File system choice is particularly difficult on Linux because there are
so many file system choices, and none of them are optimal: ext2 is not
entirely crash-safe, ext3 and xfs are journal-based, and Reiser is
optimized for small files. Fortunately, the journaling file systems
aren't significantly slower than ext2 so they are probably the best
choice.

so I don't specifically recommend ext3 anymore. As I remember, ext3 is
good only in that it can read ext2 file systems. I think XFS may be the
best bet.

Can anyone clarify if "data=writeback" is safe for PostgreSQL.
Specifically, are the data files recovered properly or is this option
only for a filesystem containing WAL?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Attachment Content-Type Size
unknown_filename text/plain 0 bytes

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Neil Conway 2002-09-26 20:41:49 Re: [HACKERS] Performance while loading data and indexing
Previous Message Jim Mercer 2002-09-26 18:54:45 PHP-4.2.3 patch to allow restriction of database access

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2002-09-26 20:27:43 WAL shortcoming causes missing-pg_clog-segment problem
Previous Message Zeugswetter Andreas SB SD 2002-09-26 18:51:04 Re: AIX compilation problems (was Re: Proposal ...)

Browse pgsql-performance by date

  From Date Subject
Next Message Neil Conway 2002-09-26 20:41:49 Re: [HACKERS] Performance while loading data and indexing
Previous Message scott.marlowe 2002-09-26 18:41:55 Re: [HACKERS] Performance while loading data and indexing