Skip site navigation (1) Skip section navigation (2)

Re: 7 hrs for a pg_restore?

From: Matthew <matthew(at)flymine(dot)org>
To: pgsql-performance(at)postgresql(dot)org
Subject: Re: 7 hrs for a pg_restore?
Date: 2008-02-20 18:18:23
Message-ID: Pine.LNX.4.64.0802201811330.20402@aragorn.flymine.org (view raw or flat)
Thread:
Lists: pgsql-performance
On Wed, 20 Feb 2008, Jeff Davis wrote:
> However, building indexes in parallel would allow better CPU
> utilization.

We have a process here that dumps a large quantity of data into an empty 
database, much like pg_restore, and then creates all the indexes at the 
end. In order to speed up that bit, I initially made it spawn off several 
threads, and make each thread run a CREATE INDEX operation in parallel. 
However, this resulted in random errors from Postgres - something to do 
with locked tables. So I changed it so that no two threads create indexes 
for the same table at once, and that solved it.

Obviously creating several indexes for the same table in parallel is 
better from a performance point of view, but you may have to fix that 
error if you haven't already.

Matthew

-- 
for a in past present future; do
  for b in clients employers associates relatives neighbours pets; do
  echo "The opinions here in no way reflect the opinions of my $a $b."
done; done

In response to

Responses

pgsql-performance by date

Next:From: Jeff DavisDate: 2008-02-20 18:35:32
Subject: Re: 7 hrs for a pg_restore?
Previous:From: Jeff DavisDate: 2008-02-20 18:04:47
Subject: Re: 7 hrs for a pg_restore?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group