Re: GPUSort project

On Wed, 2006-04-12 at 10:00 -0700, Mischa Sandberg wrote:
> Martijn van Oosterhout wrote:
> > On Tue, Apr 11, 2006 at 04:02:07PM -0700, Mischa Sandberg wrote:
> >
> >>Anybody on this list hear/opine anything pf the GPUSort project for
> >>postgresql? I'm working on a radix-sort subcase for tuplesort, and there
> >>are similarities.
> >>
> >>http://www.andrew.cmu.edu/user/ngm/15-823/project/
> >
> > I've heard it meantioned, didn't know they'd got it working. However,
> > none of my database servers have a 3D graphics anywhere near the power
> > they suggest in the article.
> >
> > Is this of practical use for run-of-the-mill video cards?
>
> Short answer: maybe.
>
> Long answer: we're shipping a server (appliance) product built on stock
> rackmount hardware, that includes an ATI Rage (8MB) with nothing to do. Much of
> what the box does is a single cpu-bound process, sorting maillog extracts. The
> GPU is an asset, even at 8MB; the headwork is in mapping/truncating sort keys
> down to dense ~32bit prefixes; and in making smooth judgements as to when to
> give the job to (a) the GPU (b) quicksort (c) a tiny bitonic sort in the SSE2
> registers.

There's been talk for the last few years in academic circles about
trying to use graphics APIs and/or specialised hardware to improve
various aspects of database technology.

It sounds like its possible, but it would have to give incredible gains
before its worth the effort to make it happen. 8MB of video RAM doesn't
score much against 256MB of normal RAM, which is pretty cheap these
days.

The hardware dependency would make this extremely sensitive to change,
so effort in this area might not give lasting benefit. As it happens,
I'm in favour of making code changes to exploit hardware, but this one
is too far for me to encourage anybody to pursue it further.

> Any of this would apply to postgres, if tuplesort.c can tolerate a preprocessing
> step that looks for special cases, and degrades gracefully into the standard
> case.

For other techniques, I think it can, depending upon the cost of the
preprocessing step. But the overall improvement from improving small
sorts could well be lost in the noise...so maybe not worth it.

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com/