Re: Sustained inserts per sec ... ?

On Apr 1, 2005 3:59 PM, Christopher Petrilli <petrilli(at)gmail(dot)com> wrote:
> On Apr 1, 2005 3:53 PM, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
> >
> > > What seems to happen is it slams into a "wall" of some sort, the
> > > system goes into disk write frenzy (wait=90% CPU), and eventually
> > > recovers and starts running for a while at a more normal speed. What
> > > I need though, is to not have that wall happen. It is easier for me
> > > to accept a constant degredation of 5%, rather than a 99% degredation
> > > for short periods, as it can cause cascade problems in the system.
> >
> > Could this possibly be a checkpoint happening?
> >
> > Also how many checkpoint segments do you have?
>
> Changes to the postgresql.conf file from "default":
>
> maintenance_work_mem = 131072
> fsync = false
> checkpoint_segments = 32

I've now had a chance to run a couple more tests, and here's two
graphs of the time required to insert (via COPY from a file) 500
records at a time:

http://www.amber.org/~petrilli/diagrams/pgsql_copy500.png
http://www.amber.org/~petrilli/diagrams/pgsql_copy500_bgwriter.png

The first is with the above changes, the second contains two
additional modificiations to the configuration:

bgwriter_percent = 25
bgwriter_maxpages = 1000

To my, likely faulty, intuition, it would seem that there is a backup
happening in the moving of data from the WAL to the final resting
place, and that by increasing these I could pull that forward. As you
can see from the charts, that doesn't seem to have any major impact.
The point, in the rough middle, is where the program begins inserting
into a new table (inherited). The X axis is the "total" number of rows
inserted. The table has:

* 21 columns (nothing too strange)
* No OIDS
* 5 indexes, including the primary key on a string

They are created by creating a main table, then doing:

CREATE TABLE foo001 INHERITS (foos);

And then recreating all the indexes.

Thoughts? Any advice would be more than appreciated.

Chris
--
| Christopher Petrilli
| petrilli(at)gmail(dot)com