Skip site navigation (1) Skip section navigation (2)

Re: CREATE INDEX as bottleneck

From: Kenneth Marshall <ktm(at)rice(dot)edu>
To: Marc Mamin <M(dot)Mamin(at)intershop(dot)de>
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: CREATE INDEX as bottleneck
Date: 2010-11-11 13:58:00
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-performance
On Thu, Nov 11, 2010 at 02:41:12PM +0100, Marc Mamin wrote:
> Hello,
> in the last years, we have successfully manage to cope with our data
> growth 
> using partitioning and splitting large aggregation tasks on multiple
> threads.
> The partitioning is done logically by our applicationn server, thus
> avoiding trigger overhead.
> There are a few places in our data flow where we have to wait for index
> creation before being able to distribute the process on multiple threads
> again.
> With the expected growth, create index will probably become a severe
> bottleneck for us.
> Is there any chance to see major improvement on it in a middle future ?
> I guess the question is naive, but why can't posgres use multiple
> threads for large sort operation ?
> best regards,
> Marc Mamin

There has been a recent discussion on the hackers mailing list on
using the infrastructure that is already in place to lauch autovacuum
processes to launch other helper processes. Something like this could
be used to offload the sort process to a much more parallelize version
that could take advantage of multiple I/O streams and CPU cores. Many
things are possible given the appropriate resources: funding, coding
and development cycles...


In response to

pgsql-performance by date

Next:From: Mladen GogalaDate: 2010-11-11 14:15:58
Subject: Re: anti-join chosen even when slower than old plan
Previous:From: Kenneth MarshallDate: 2010-11-11 13:51:22
Subject: Re: anti-join chosen even when slower than old plan

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group