From:
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To:
Simon Riggs <simon(at)2ndQuadrant(dot)com>
Cc:
Jeff Janes <jeff(dot)janes(at)gmail(dot)com>,
pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject:
Re: [PERFORM] Slow query: bitmap scan troubles
Date:
2013-01-07 18:27:38
Message-ID:
13842.1357583258@sss.pgh.pa.us (view raw or flat )
Thread:
2012-12-04 15:06:48 from <postgresql(at)foo(dot)me(dot)uk>
2012-12-04 15:21:17 from <postgresql(at)foo(dot)me(dot)uk>
2012-12-04 15:27:57 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2012-12-04 17:22:29 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2012-12-04 17:25:56 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2012-12-04 17:35:32 from "Philip Scott" <pscott(at)foo(dot)me(dot)uk>
2012-12-04 17:47:29 from <postgresql(at)foo(dot)me(dot)uk>
2012-12-04 18:50:41 from Vitalii Tymchyshyn <tivv00(at)gmail(dot)com>
2012-12-04 18:55:17 from "Philip Scott" <pscott(at)foo(dot)me(dot)uk>
2012-12-04 18:56:04 from <postgresql(at)foo(dot)me(dot)uk>
2012-12-04 20:11:53 from Sergey Konoplev <gray(dot)ru(at)gmail(dot)com>
2012-12-06 12:56:26 from <postgresql(at)foo(dot)me(dot)uk>
2012-12-04 22:34:42 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2012-12-04 18:03:29 from <postgresql(at)foo(dot)me(dot)uk>
2012-12-04 18:31:50 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2012-12-04 23:42:21 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2012-12-05 17:39:35 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2012-12-05 17:43:49 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2012-12-06 17:27:48 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2012-12-06 20:05:09 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2012-12-06 21:09:54 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2012-12-05 18:05:10 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2012-12-06 12:52:07 from <postgresql(at)foo(dot)me(dot)uk>
2012-12-18 06:00:16 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2013-01-05 22:18:16 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-06 16:29:17 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2013-01-06 18:18:03 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-06 18:22:33 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2013-01-06 18:19:10 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2013-01-06 18:58:40 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-06 19:47:48 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2013-01-06 23:03:05 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-06 23:17:23 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-07 00:03:04 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2013-01-07 17:35:51 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-07 18:03:37 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2013-01-07 18:27:38 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-07 18:48:12 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2013-01-11 01:07:34 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-14 16:45:01 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-14 17:23:17 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-14 17:50:24 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-14 17:56:37 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-15 19:46:39 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-15 20:11:02 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2012-12-19 01:05:05 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2012-12-19 14:40:54 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2012-12-06 14:10:29 from <postgresql(at)foo(dot)me(dot)uk>
2012-12-08 15:15:42 from Guillaume Lelarge <guillaume(at)lelarge(dot)info>
2012-12-10 09:52:42 from <postgresql(at)foo(dot)me(dot)uk>
2012-12-04 18:31:05 from "Philip Scott" <pscott(at)foo(dot)me(dot)uk>
2012-12-04 18:32:57 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2012-12-04 18:54:29 from <postgresql(at)foo(dot)me(dot)uk>
Lists:
pgsql-hackers pgsql-performance
Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
> On 7 January 2013 17:35, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> That gives a formula of
>> cpu_operator_cost * log2(N) + cpu_operator_cost * 50 * (H+2)
> Again, this depends on N and H, so thats good.
> I think my retinas detached while reading your explanation, but I'm a
> long way from coming up with a better or more principled one.
> If we can describe this as a heuristic that appears to fit the
> observed costs, we may keep the door open for something better a
> little later.
I'm fairly happy with the general shape of this formula: it has a
principled explanation and the resulting numbers appear to be sane.
The specific cost multipliers obviously are open to improvement based
on future evidence. (In particular, I intend to code it in a way that
doesn't tie the "startup overhead" and "cost per page" numbers to be
equal, even though I'm setting them equal for the moment for lack of a
better idea.)
One issue that needs some thought is that the argument for this formula
is based entirely on thinking about b-trees. I think it's probably
reasonable to apply it to gist, gin, and sp-gist as well, assuming we
can get some estimate of tree height for those, but it's obviously
hogwash for hash indexes. We could possibly just take H=0 for hash,
and still apply the log2(N) part ... not so much because that is right
as because it's likely too small to matter.
regards, tom lane
In response to
Responses
pgsql-performance by date
Next :From: ktm@rice.eduDate: 2013-01-07 18:28:11
Subject : Re: Sub optimal performance with default setting of
Postgresql with FreeBSD 9.1 on ZFS
Previous :From : nobody nowhereDate : 2013-01-07 18:10:17
Subject : Re: SMP on a heavy loaded database FIXED !!!!
pgsql-hackers by date
Next :From: Claudio FreireDate: 2013-01-07 18:48:12
Subject : Re: [PERFORM] Slow query: bitmap scan troubles
Previous :From : Simon RiggsDate : 2013-01-07 18:03:37
Subject : Re: [PERFORM] Slow query: bitmap scan troubles