| From: | Greg Stark <stark(at)mit(dot)edu> |
|---|---|
| To: | Josh Berkus <josh(at)agliodbs(dot)com> |
| Cc: | Peter Geoghegan <peter(dot)geoghegan86(at)gmail(dot)com>, Simon Riggs <simon(at)2ndquadrant(dot)com>, Jeff Janes <jeff(dot)janes(at)gmail(dot)com>, Merlin Moncure <mmoncure(at)gmail(dot)com>, postgres performance list <pgsql-performance(at)postgresql(dot)org> |
| Subject: | Re: Yet another abort-early plan disaster on 9.3 |
| Date: | 2014-10-10 11:16:08 |
| Message-ID: | CAM-w4HNfZymQmTu3+TxQQD-e6_410-sDnZBaW8neurmTFh4GbA@mail.gmail.com |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers pgsql-performance |
On Thu, Oct 2, 2014 at 8:56 PM, Josh Berkus <josh(at)agliodbs(dot)com> wrote:
> Yes, it's only intractable if you're wedded to the idea of a tiny,
> fixed-size sample. If we're allowed to sample, say, 1% of the table, we
> can get a MUCH more accurate n_distinct estimate using multiple
> algorithms, of which HLL is one. While n_distinct will still have some
> variance, it'll be over a much smaller range.
I've gone looking for papers on this topic but from what I read this
isn't so. To get any noticeable improvement you need to read 10-50% of
the table and that's effectively the same as reading the entire table
-- and it still had pretty poor results. All the research I could find
went into how to analyze the whole table while using a reasonable
amount of scratch space and how to do it incrementally.
--
greg
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Thom Brown | 2014-10-10 11:25:59 | Re: Column Redaction |
| Previous Message | Heikki Linnakangas | 2014-10-10 11:15:43 | Re: Column Redaction |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tomas Vondra | 2014-10-10 12:10:13 | Re: Yet another abort-early plan disaster on 9.3 |
| Previous Message | Emi Lu | 2014-10-08 14:42:55 | Re: char(N), varchar(N), varchar, text |