Skip site navigation (1) Skip section navigation (2)

pgsql inserts problem

From: Tarhon-Onu Victor <mituc(at)iasi(dot)rdsnet(dot)ro>
To: pgsql-bugs(at)postgresql(dot)org
Cc: pgsql-performance(at)postgresql(dot)org
Subject: pgsql inserts problem
Date: 2003-08-27 12:50:32
Message-ID: Pine.LNX.4.55.0308271534120.21841@blackblue.iasi.rdsnet.ro (view raw or flat)
Thread:
Lists: pgsql-bugspgsql-performance
	Hi,

	I have a (big) problem with postgresql when making lots of 
inserts per second. I have a tool that is generating an output of ~2500 
lines per seconds. I write a script in PERL that opens a pipe to that 
tool, reads every line and inserts data.
	I tryed both commited and not commited variants (the inserts 
were commited at every 60 seconds), and the problem persists.

	The problems is that only ~15% of the lines are inserted into 
the database. The same script modified to insert the same data in a 
similar table created in a MySQL database inserts 100%.

	I also dropped the indexes on various columns, just to make sure 
that the overhead is not to big (but I also need that indexes because 
I'll make lots of SELECTs from that table).
	I tried both variants: connecting to a host and localy (through 
postgresql server's socket (/tmp/s.PGSQL.5432).

	Where can be the problem?

	I'm using postgresql 7.4 devel snapshot 20030628 and 20030531. 
Some of the settings are:

shared_buffers = 520
max_locks_per_transaction = 128
wal_buffers = 8 
max_fsm_relations = 30000
max_fsm_pages = 482000 
sort_mem = 131072
vacuum_mem = 131072
effective_cache_size = 10000
random_page_cost = 2

-- 
Any views or opinions presented within this e-mail are solely those of
the author and do not necessarily represent those of any company, unless
otherwise expressly stated.

Responses

pgsql-performance by date

Next:From: mattDate: 2003-08-27 13:17:44
Subject: Re: Hardware recommendations to scale to silly load
Previous:From: Tomka GergelyDate: 2003-08-27 12:40:02
Subject: Re: Tests

pgsql-bugs by date

Next:From: mikeDate: 2003-08-27 15:43:50
Subject: Re: index not used afer VACUUM ANALYZE
Previous:From: Keow Yeong Huat JosephDate: 2003-08-27 09:43:59
Subject: Re: Install postgresql 7.3 on Solaris 2.6 sparc

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group