Re: Performance while loading data and indexing

Hi Shridhar,

Shridhar Daithankar wrote:
<snip>
> 3) Any suggsestions for runtime as data load and query will be going in
> parallel.

That sounds unusual. From reading this, it *sounds* like you'll be
running queries against an incomplete dataset, or maybe just running the
queries that affect the tables loaded thus far (during the initial
load).

<snip>
> fsync=true (Sad but true. Left untouched.. Will that make a difference on
> SCSI?)

Definitely. Have directly measured a ~ 2x tps throughput increase on
FreeBSD when leaving fsync off whilst performance measuring stuff
recently (PG 7.2.2). Like anything it'll depend on workload, phase of
moon, etc, but it's a decent indicator.

<snip>
> Now questions.
>
> 1) Instead of copying from a single 12GB data file, will a parallel copy from
> say 5 files will speed up the things?

Not sure yet. Haven't get done enough performance testing (on the cards
very soon though).

> Couple MB of data per sec. to disk is just not saturating it. It's a RAID 5
> setup..

fsync = off would help during the data load, but not a good idea if
you're going to be running queries against it at the same time.

Am still getting the hang of performance tuning stuff. Have a bunch of
Ultra160 hardware for the Intel platform, and am testing against it as
time permits.

Not as high end as I'd like, but it's a start.

:-)

Regards and best wishes,

Justin Clift

<snip>
> Bye
> Shridhar

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi