Skip site navigation (1) Skip section navigation (2)

Re: Concurrent COPY commands

From: Alan Hodgson <ahodgson(at)simkin(dot)ca>
To: pgsql-novice(at)postgresql(dot)org
Subject: Re: Concurrent COPY commands
Date: 2008-07-02 17:02:48
Message-ID: 200807021002.48627@hal.medialogik.com (view raw or flat)
Thread:
Lists: pgsql-novice
On Wednesday 02 July 2008, Phillip Sitbon <phillip(at)sitbon(dot)net> wrote:
> Hello,
>
> I am running some queries that use multiple connections to issue COPY
> commands which bring data into the same table via different files (FIFOs
> to be precise). This is being done on a SMP machine and I am noticing
> that none of the postgres worker processes operate in parallel, even
> though there is data available to all of them. The performance is nearly
> exactly the same as it is for issuing a single COPY command.
> Is this 
> normal behavior, even with all of the separate transactions still in
> progress? Would I be better off doing multithreaded bulk inserts from my
> C program rather than sending the data to FIFOs?

Sounds like you're I/O bound - I doubt any other concurrency mechanism will 
change that much.

>
> The machine I am using has 16GB of memory and 8 cores, so I've tried to
> optimize the configuration accordingly but I am a little lost in some
> places.

Ah, but what does your RAID controller and drives look like?


-- 
Alan

In response to

Responses

pgsql-novice by date

Next:From: richard terryDate: 2008-07-03 05:09:10
Subject: date formatting question
Previous:From: Tom LaneDate: 2008-07-02 17:01:59
Subject: Re: Calculating repeating events - functionality lost with the demise of tinterval ?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group