Re: realtime data inserts

From: Adam Siegel <adam(at)sycamore(dot)us>
To: "Jim C(dot) Nasby" <jim(at)nasby(dot)net>
Cc: pgsql-hackers(at)postgresql(dot)org, <pgsql-performance(at)postgresql(dot)org>, <pgsql-general(at)postgresql(dot)org>
Subject: Re: realtime data inserts
Date: 2003-05-12 14:51:17
Message-ID: Pine.LNX.4.44.0305121048021.6628-100000@beorn.hq.sai
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers pgsql-performance


The copy from method (PQputline) allows me to achieve around 1000 inserts
per second.

On Sat, 10 May 2003, Jim C. Nasby wrote:

> Are you binding your insert? IE:
>
> prepare statement INSERT INTO blah VALUES (?, ?, ?);
>
> execute statement (a, b, c)
>
> Instead of just "INSERT INTO blah VALUES(a, b, c)"
>
>
> On Sat, May 10, 2003 at 11:25:16AM -0400, Adam Siegel wrote:
> > I have realtime data flowing at a rate of 500, 512 byte packets per second.
> > I want to log the info in a database table with two other columns, one for a
> > timestamp and one for a name of the packet. The max rate I can achieve is
> > 350 inserts per second on a sun blade 2000. The inserts are grouped in a
> > transaction and I commit every 1200 records. I am storing the binary data
> > in a bytea. I am using the libpq conversion function. Not sure if that is
> > slowing me down. But I think it is the insert not the conversion.
> >
> > Any thoughts on how to achive this goal?
>
>
>

In response to

Browse pgsql-general by date

  From Date Subject
Next Message scott.marlowe 2003-05-12 14:58:16 Re: Zope
Previous Message Max Bernaert 2003-05-12 14:30:31 Re: More than one user on postgresql database on Windows ?

Browse pgsql-hackers by date

  From Date Subject
Next Message Lamar Owen 2003-05-12 14:56:02 Re: PostgreSQL Cookbook Site Down
Previous Message Tom Lane 2003-05-12 13:44:12 Re: SET CONSTRAINTS not schema-aware

Browse pgsql-performance by date

  From Date Subject
Next Message Josh Berkus 2003-05-12 15:49:39 Re: PERFORMANCE and SIZE
Previous Message Lucas Adamski 2003-05-12 07:29:50 Re: Hack around lack of CORRESPONDING BY in EXCEPT?