Re: Thoughts on statistics for continuously advancing columns

On 31/12/2009 12:33 AM, Kevin Grittner wrote:
> Tom Lane<tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
>> Well, the problem Josh has got is exactly that a constant high
>> bound doesn't work.
>
> I thought the problem was that the high bound in the statistics fell
> too far below the actual high end in the data. This tends (in my
> experience) to be much more painful than an artificially extended
> high end in the statistics. (YMMV, of course.)
>
>> What I'm wondering about is why he finds that re-running ANALYZE
>> isn't an acceptable solution. It's supposed to be a reasonably
>> cheap thing to do.
>
> Good point. We haven't hit this problem in PostgreSQL precisely
> because we can run ANALYZE often enough to prevent the skew from
> becoming pathological.

While regular ANALYZE seems to be pretty good ... is it insane to
suggest determining the min/max bounds of problem columns by looking at
a btree index on the column in ANALYZE, instead of relying on random
data sampling? An ANALYZE that didn't even have to scan the indexes but
just look at the ends might be something that could be run much more
frequently with less I/O and memory cost than a normal ANALYZE, just to
selectively update key stats that are an issue for such continuously
advancing columns.

--
Craig Ringer