WIP: Hash Join-Filter Pruning using Bloom Filters

From: "Jonah H(dot) Harris" <jonah(dot)harris(at)gmail(dot)com>
To: "Pg Hackers" <pgsql-hackers(at)postgresql(dot)org>
Subject: WIP: Hash Join-Filter Pruning using Bloom Filters
Date: 2008-11-02 21:49:24
Message-ID: 36e682920811021349h7202bdecpd7a45c8a8038465e@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

All,

Attached is an initial patch I've been playing with which uses Bloom
filters to reduce unnecessary processing of outer tuples in hash
joins. In short, this works by creating a Bloom filter, adding all
relevant tuples for the inner relation, and querying the filter (for
existence) when retrieving tuples from the outer relation. This
avoids unnecessary tuple movement and bucket searches for matches we
already know can't exist. Currently it works only for JOIN_INNER, but
could be modified to optimize anti/semi joins as well. Similarly, I
created a GUC to enable pruning, named bloom_pruning.

Rather than performing k hash functions, this implementation simply
sets a bit based on the already-computed hash value. I wanted to send
this around for reviews and comments before working on it further. As
this isn't overly intrusive, if someone can commit to reviewing and
providing input, I'll commit to having this ready for 8.4.

--
Jonah H. Harris, Senior DBA
myYearbook.com

Attachment Content-Type Size
bloompruning_v1.patch application/octet-stream 9.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Hannes Eder 2008-11-02 22:36:53 Re: WIP: Hash Join-Filter Pruning using Bloom Filters
Previous Message Tom Lane 2008-11-02 21:27:19 Re: don't use MAKE_PTR/OFFSET for shmem pointers