Re: Threaded Sorting

On Fri, 4 Oct 2002, Bruce Momjian wrote:

> Hans-Jürgen Schönig wrote:
> > Did anybody think about threaded sorting so far?
> > Assume an SMP machine. In the case of building an index or in the case
> > of sorting a lot of data there is just one backend working. Therefore
> > just one CPU is used.
> > What about starting a thread for every temporary file being created?
> > This way CREATE INDEX could use many CPUs.
> > Maybe this is worth thinking about because it will speed up huge
> > databases and enterprise level computing.
>
> We haven't thought about it yet because there are too many buggy thread
> implementations. We are probably just now getting to a point where we
> can consider it. However, lots of databases have moved to threads for
> all sorts of things and ended up with a royal mess of code. Threads
> can only improve things in a few areas of the backend so it would be
> nice if we could limit the exposure to threads to those areas; sorting
> could certainly be one of them, but frankly, I think disk I/O is our
> limiting factore there. I would be interested to see some tests that
> showed otherwise.

Wouldn't the type of disk subsystem really make a big difference here?

With a couple of U160 cards and a dozen 15krpm hard drives, I would
imagine I/O would no longer be as much of an issue as a single drive
system would be.

It seems like sometimes we consider these issues more from the one or two
SCSI drives perspective insted of the big box o drives perspective.