Skip site navigation (1) Skip section navigation (2)

Performance tuning for copy in

From: Joe Maldonado <jmaldonado(at)webehosting(dot)biz>
To: pgsql-admin(at)postgresql(dot)org
Subject: Performance tuning for copy in
Date: 2004-04-14 14:54:22
Message-ID: 1081954461.3941.17.camel@joem (view raw or flat)
Thread:
Lists: pgsql-admin
Hello,
Currently I can get no more than 40k records/second to be copied
in to postgres.  This is also the case when running multiple
cocurrent copy in commands from different connections.  It
seems linear in that 1 process will copy in at 40k/s and 2
will copy in at about 20k/s, etc.

I need to know whether I can get better performance from
the db with the hardware specified below.  Changing from
mirrored to stripe RAID is not an option :( at this time
as is turning off fsync (which did not work much better).


-Joe Maldonado

Configuration:
PostgreSQL 7.4 
configure --libdir=/usr/lib \
          --includedir=/usr/include \
          --bindir=/usr/bin \
          --with-krb5=/usr/kerberos \
          --with-openssl=/usr \
          --enable-thread-safety \
          --enable-debug

OS: RedHat 9
Drive Config:
sdb1: /pg_clog
sdb2: /pg_xlog
sdc1: /pg_data (RAID 1)

hdparm output shows average of 45MB/s
iostat shows fairly constant 10MB/s written to pg_data
and 5-8MB/s to WAL.

CPU: 2x 2.0G Xeon



pgsql-admin by date

Next:From: Bruno Wolff IIIDate: 2004-04-14 14:57:53
Subject: Re: Real time replication of PG DBs accross two servers -
Previous:From: Ben KimDate: 2004-04-14 14:34:10
Subject: Re: sequence value of the record just inserted.

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group