Re: WIP: Hash Join-Filter Pruning using Bloom Filters

On Mon, Nov 10, 2008 at 2:42 PM, Lawrence, Ramon <ramon(dot)lawrence(at)ubc(dot)ca> wrote:
> I have tested the Bloom filter patch. It compiles cleanly against HEAD.

Thank you for testing this!

> As indicated, the performance improvements for hash join are good,
> especially when the build table is filtered with a selection condition.
> Performance improvements range from a couple of percent up to 20% for
> multi-batch joins. Note that the bloom filter will slightly slow
> queries where the filter has no benefit.

I have a new patch which does not create a bloom filter unless it sees
that the hash join is going to batch. I'll send it along later
tonight.

> I have not looked at the actual implementation of the Bloom filter, but
> will proceed to do that next. One issue to be considered is how the
> space used for the bloom filter is related to the work_mem allocated to
> the join. That is, does the bloom filter consume some of the work_mem
> space or is it treated as additional memory allocated to the join.

Currently it's additional space not accounted for by work_mem.
Additionally, it's a good amount more space than is required. This is
fixed in the newer patch as well.

--
Jonah H. Harris, Senior DBA
myYearbook.com