Re: would hw acceleration help postgres (databases in general) ?

Yes!

Probably very much so. There is good evidence that using multiple CPU's and
GPU's will
speed sorting and many other database operations too.

See
http://www.cs.cmu.edu/afs/cs.cmu.edu/Web/People/ngm/15-823/project/Final.pdf

The question become how practical is it? There are numerous issues; Sorts
often use multiple
columns and character data as keys. Little research has been done on
sorting multi-column,
variable length character data types on GPUs. Most research papers have
used a single numeric
(int or real) key. Fixed length character encodings such as UCS-2 or
UCS-4 and not UTF-8
or UTF-16 will work faster, at the expense of storage, for character data
sorting,

For PostgreSQL you also need to support many platforms. Most database GPGPU
research
has been done using Nvidia's CUDA programming environment.
Several papers have been published on using CUDA with PostgreSQL.
But do we want to be tied into a single vendors hw?

No, Never!

OpenCL, is C Language for programming GPU's and many-core CPUs.
It addresses the platform problem; it runs on Windows, MAC, and Linux.
It supports Nvidia, and AMD GPU's. It supports Intel, AMD, and ARM
CPU's and IBM/Toshiba Cell processors too.

Most GPU programming models are data array centric. They work great if
your data type is a
naturally array centric like vectors, matrices, or images. They also work
better if the data is fixed length too.
SQL is relational; row and column centric. SQL data types are not usually
fixed length vectors, arrays or images.
But most SQL data types can be handled with OpenCL on GPUs with some work.

To leverage GPUs well, you need good parallel algorithms that effectively
utilize the GPU's
memory model. These parallel algorithms will give good speed-ups, based
on Amdahl's Law, only if they are very parallel.
E.g. 99% or better. So there lots of work to do in algorithms and their
practical implementation.

There is great potential in hw accelerated databases. This is why I've
started PgOpenCL language bindings project
See
http://www.scribd.com/doc/51484335/PostgreSQL-OpenCL-Procedural-Language-pgEast-March-2011

There a still more issues like multi-tasking, fixed memory size, data
transfer speed to resolve.
Using GPUs won't solve data I/O parallelism problems. But once the data is
in RAM they can kick some serious ...
We need to research and conquer the issues that hold us back from using hw
acceleration like GPU's.
Running OpenCL inside PostgreSQL is just a first step.

And I can't believe I've made a post on this subject without bringing up the
topic of multi-threading...
It's price you have to pay to utilize the 1400+ Arithmetic Logic Units on
high end GPU.
Each ALU can evaluate a predicate, or sum 2 numbers, and do lots more in
parallel.

With OpenCL, MT is built into the language, you don't have to mess with
P-threads, and such evils.
OpenCL has some simple built-in synchronization techniques, that's a
fundamental for parallel programming too.

-Tim

--
View this message in context: http://postgresql.1045698.n5.nabble.com/would-hw-acceleration-help-postgres-databases-in-general-tp3301110p4266108.html
Sent from the PostgreSQL - hackers mailing list archive at Nabble.com.