From: | "AV" <avek_nospam_(at)videotron(dot)ca> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | Re: Thread or not threads? |
Date: | 2001-05-02 11:54:12 |
Message-ID: | sLSH6.655$d_.255621@weber.videotron.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
If your driver jdbc2.0 compliant, look at
batch processing. Choice between
one connection and many connections is not
obvious. Good driver is supposed to handle
many processes per one connection.
I would suggest starting with connection pool and 10 threads
with 10 connections doing batch update by chunks of
100 records. Than you can optimize parameters...
AlexV
"Raymond Chui" <raymond(dot)chui(at)noaa(dot)gov> wrote in message
news:3AEFECE4(dot)C51252CC(at)noaa(dot)gov(dot)(dot)(dot)
>
>
> I have 4 columns in a table, id, sub_id, timestamp and value.
> The primary key is id, sub_id and timestamp combine.
> I need to insert many rows (may be 10 thousands every 4 minutes)
> as fast as I can to the same host, same port, same database, same table.
>
> A.
> Open only one JDBC (Java Database Connective) connection,
> have multiple threads (similar to UNIX child process) to do
> the insert.
> Note, too many threads will cause the system out of memory!
>
> B.
> Open only one JDBC connection, have only one single thread
> to do the insert.
>
> C.
> Open multiple JDBC connections threads, each one of them
> handle the data insert.
>
> D.
> Please tell me your way, the much better way.
>
> Currently I am doing A. I have to limited the number of threads that
> won't cause the system out of memory. But there is big bottleneck there.
>
> The UNIX system scheduling can only do one insert a time (I think).
> I am wonder choose C will be better? Tell me the D.
> Thank you very much in advance!
>
> P.S.
> The database server is PostgreSQL 7.x. and the UNIX box is Redhat Linux
> 6.5.
>
> --
> Why we want to teach our babies to talk and walk,
> then later we tell them "sit down!", "be quiet!" ?
>
> Democracy is not a better way for a solution,
> it is just another way to spread the blames.
>
> --Raymond
>
>
From | Date | Subject | |
---|---|---|---|
Next Message | Carl Rosenberger | 2001-05-02 13:40:35 | Re: Thread or not threads? |
Previous Message | Raymond Chui | 2001-05-02 11:17:56 | Thread or not threads? |