Re: [HACKERS] choose_bitmap_and again (was Re: [PERFORM] Strangely Variable Query Performance)

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-patches(at)postgreSQL(dot)org
Cc: Alvaro Herrera <alvherre(at)commandprompt(dot)com>, Steve <cheetah(at)tanabi(dot)org>
Subject: Re: [HACKERS] choose_bitmap_and again (was Re: [PERFORM] Strangely Variable Query Performance)
Date: 2007-04-14 22:02:53
Message-ID: 23414.1176588173@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches pgsql-performance

I wrote:
> Thinking more about this leads me to the following proposal:

> 1. Explicitly group the indexes according to the subset of
> WHERE-conditions (and partial index conditions, if any) they use.
> Within each such group, discard all but the cheapest-scan-cost one.

> 2. Sort the remaining indexes according to scan cost.

> 3. For each index in order, consider it as a standalone scan, and also
> consider adding it on to the AND-group led by each preceding index,
> using the same logic as now: reject using any WHERE-condition twice
> in a group, and then add on only if the total cost of the AND-group
> scan is reduced.

Here is a patch along these lines, in fact two patches (HEAD and 8.2
versions). The 8.2 version picks up some additional partial-index
intelligence that I added to HEAD on Mar 21 but did not at that time
risk back-patching --- since this is a fairly large rewrite of the
routine, keeping the branches in sync seems best.

Steve, can you try this out on your queries and see if it makes better
or worse decisions? It seems to fix your initial complaint but I do
not have a large stock of test cases to try.

regards, tom lane

Attachment Content-Type Size
choose-bitmap-and-head.patch.gz application/octet-stream 6.5 KB
choose-bitmap-and-82.patch.gz application/octet-stream 6.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Zoltan Boszormenyi 2007-04-14 22:44:22 Re: IDENTITY/GENERATED v36 Re: Final version of IDENTITY/GENERATED patch
Previous Message Andrew Dunstan 2007-04-14 19:27:56 Re: build/install xml2 when configured with libxml

Browse pgsql-patches by date

  From Date Subject
Next Message Zoltan Boszormenyi 2007-04-14 22:44:22 Re: IDENTITY/GENERATED v36 Re: Final version of IDENTITY/GENERATED patch
Previous Message Andrew Dunstan 2007-04-14 19:27:56 Re: build/install xml2 when configured with libxml

Browse pgsql-performance by date

  From Date Subject
Next Message Steve 2007-04-14 22:55:45 Re: [HACKERS] choose_bitmap_and again (was Re: [PERFORM] Strangely Variable Query Performance)
Previous Message Tom Lane 2007-04-14 18:21:26 Re: [HACKERS] choose_bitmap_and again (was Re: Strangely Variable Query Performance)