Skip site navigation (1)
Skip section navigation (2)
## pgsql: Fix misuse of Lossy Counting (LC) algorithm in

### pgsql-committers by date

Log Message: ----------- Fix misuse of Lossy Counting (LC) algorithm in compute_tsvector_stats(). We must filter out hashtable entries with frequencies less than those specified by the algorithm, else we risk emitting junk entries whose actual frequency is much less than other lexemes that did not get tabulated. This is bad enough by itself, but even worse is that tsquerysel() believes that the minimum frequency seen in pg_statistic is a hard upper bound for lexemes not included, and was thus underestimating the frequency of non-MCEs. Also, set the threshold frequency to something with a little bit of theory behind it, to wit assume that the input distribution is approximately Zipfian. This might need adjustment in future, but some preliminary experiments suggest that it's not too unreasonable. Back-patch to 8.4, where this code was introduced. Jan Urbanski, with some editorialization by Tom Modified Files: -------------- pgsql/src/backend/tsearch: ts_typanalyze.c (r1.8 -> r1.9) (http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/tsearch/ts_typanalyze.c?r1=1.8&r2=1.9)

Next: From:Tom LaneDate:2010-05-30 21:59:10Subject: pgsql: Fix misuse of Lossy Counting (LC) algorithm inPrevious: From: Tom LaneDate: 2010-05-30 18:10:41Subject: pgsql: Change the notation for calling functions with named parameters