Re: Threaded Sorting

Greg Copeland wrote:

>I wouldn't hold your breath for any form of threading. Since PostgreSQL
>is process based, you might consider having a pool of sort processes
>which address this but I doubt you'll get anywhere talking about threads
>here.
>
>Greg
>
>
>

I came across the problem yesterday. We thought about SMP and did some
tests on huge tables. The postmaster was running full speed to get the
stuff sorted - even on an IDE system.
I asked my friends who are doing a lot of work with Oracle on huge SMP
machines. I was told that Oracle has a mechanism which can run efficient
sorts on SMP machines. It seems to speed up sorting a lot.

If we could reduce the time needed to build up an index by 25% it would
be a wonderful thing. Just think of a scenario:
1 thread: 24 hours
many threads: 18 hours

We could gain 6 hours which is a LOT.
We have many people running PostgreSQL on systems having wonderful IO
systems - in this case IO is not the bottleneck anymore.

I had a brief look at the code used for sorting. It is very well
documented so maybe it is worth thinking about a parallel algorithm.

When talking about threads: A pool of processes for sorting? Maybe this
could be useful but I doubt if it the best solution to avoid overhead.
Somewhere in the TODO it says that there will be experiments with a
threaded backend. This make me think that threads are not a big no no.

Hans

--
*Cybertec Geschwinde u Schoenig*
Ludo-Hartmannplatz 1/14, A-1160 Vienna, Austria
Tel: +43/1/913 68 09; +43/664/233 90 75
www.postgresql.at <http://www.postgresql.at>, cluster.postgresql.at
<http://cluster.postgresql.at>, www.cybertec.at
<http://www.cybertec.at>, kernel.cybertec.at <http://kernel.cybertec.at>