A Question About Insertions -- Performance

From: "Clay Luther" <claycle(at)cisco(dot)com>
To: "Pgsql-General (E-mail)" <pgsql-general(at)postgresql(dot)org>
Subject: A Question About Insertions -- Performance
Date: 2003-09-10 20:02:29
Message-ID: F67EB38120F7BB4BB972C786095802070E3402@ipcbu-exchange.amer.unity.cisco.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I am doing to large dataset performance tests with 7.3.4b2 today and I noticed an interesting phenomenon. My shared memory buffers are set at 128MB. Peak postmaster usage appears to be around 90MB.

My test app performs inserts across 4 related tables, each set of 4 inserts representing a single theoretical "device" object. I report how many "devices" I have inserted, per second, for example...

[...]
41509 devices inserted, 36/sec
[1 second later]
41544 devices inserted, 35/sec
[...]

(to be clear, 41509 devices inserted equals 166036 actual, related rows in the db)

Performance follows an odd "peak and valley" pattern. It will start out with a high insertion rate (commits are performed after each "device set"), then after a few thousand device sets, performance will drop to 1 device/second for about 5 seconds. Then it will slowly ramp up over the next 10 seconds to /just below/ the previous high water mark. A few thousand inserts later, it will drop to 1 device/second again for 5 seconds, then slowly ramp up to just below the last high water mark.

Ad infinitum.

I am wondering:

1) What am I seeing here? This is on a 4-processor machine and postmaster has a CPU all to itself, so I ruled out processor contention.

2) Is there more performance tuning I could perform to flatten this out, or is this just completely normal? Postmaster never busts over 100MB out of the 128MB shared memory I've allocated to it, and according to <mumble mumble webpage mumble>, this is just about perfect for shared memory settings (100 to 120% high water mark).

Thanks.

---
Clay
Cisco Systems, Inc.
claycle(at)cisco(dot)com
(972) 813-5004

I've stopped 19,647 spam messages. You can too!
One month FREE spam protection at http://www.cloudmark.com/spamnetsig/}

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Clay Luther 2003-09-10 21:01:00 50K record DELETE Begins, 100% CPU, Never Completes 1 hour later
Previous Message Michał Zaborowski 2003-09-10 19:46:28 Re: Picture with Postgres and Delphi