Skip site navigation (1) Skip section navigation (2)

Re: Gsoc2012 idea, tablesample

From: Qi Huang <huangqiyx(at)hotmail(dot)com>
To: <heikki(dot)linnakangas(at)enterprisedb(dot)com>, <josh(at)agliodbs(dot)com>
Cc: <pgsql-hackers(at)postgresql(dot)org>, <andres(at)anarazel(dot)de>,<alvherre(at)commandprompt(dot)com>, <neil(dot)conway(at)gmail(dot)com>, <daniel(at)heroku(dot)com>,<cbbrowne(at)gmail(dot)com>, <kevin(dot)grittner(at)wicourts(dot)gov>
Subject: Re: Gsoc2012 idea, tablesample
Date: 2012-04-17 11:55:36
Message-ID: BAY159-W3319FCEE2FBB7561A1CEE9A33F0@phx.gbl (view raw or flat)
Thread:
Lists: pgsql-hackers



Hi, Heikki   Thanks for your advice.    I will change my plan accordingly. But I have a few questions.
> 1. We probably don't want the SQL syntax to be added to the grammar. 
> This should be written as an extension, using custom functions as the 
> API, instead of extra SQL syntax.
> 

1. "This should be written as an extension, using custom functions as the API". Could you explain a bit more what does this mean?   
> 2. It's not very useful if it's just a dummy replacement for "WHERE 
> random() < ?". It has to be more advanced than that. Quality of the 
> sample is important, as is performance. There was also an interesting 
> idea of on implementing monetary unit sampling.

2. In the plan, I mentioned using optimizer statistics to improve the quality of sampling. I may emphasize on that point. I will read about monetary unit sampling and add into the plan about possibility of implementing this idea. 
> Another idea that Robert Haas suggested was to add support doing a TID 
> scan for a query like "WHERE ctid< '(501,1)'". That's not enough work 
> for GSoC project on its own, but could certainly be a part of it.

3. I read about the replies on using ctid. But I don't quite understand how that might help. ctid is just a physical location of row version within the table. If I do "where ctid<'(501, 1)'", what is actually happening? Can I add in this as an optional implementation? I think I can check how to do this if I can have enough time in this project. 

Best Regards and ThanksHuang Qi VictorComputer Science Department- National University of Singapore

> Date: Tue, 17 Apr 2012 09:16:29 +0300
> From: heikki(dot)linnakangas(at)enterprisedb(dot)com
> To: josh(at)agliodbs(dot)com
> CC: huangqiyx(at)hotmail(dot)com; pgsql-hackers(at)postgresql(dot)org; andres(at)anarazel(dot)de; alvherre(at)commandprompt(dot)com; neil(dot)conway(at)gmail(dot)com; daniel(at)heroku(dot)com; cbbrowne(at)gmail(dot)com; kevin(dot)grittner(at)wicourts(dot)gov
> Subject: [HACKERS] Gsoc2012 idea, tablesample
> 
> On 24.03.2012 22:12, Joshua Berkus wrote:
> > Qi,
> >
> > Yeah, I can see that.  That's a sign that you had a good idea for a project, actually: your idea is interesting enough that people want to debate it.  Make a proposal on Monday and our potential mentors will help you refine the idea.
> 
> Yep. The discussion withered, so let me try to summarize:
> 
> 1. We probably don't want the SQL syntax to be added to the grammar. 
> This should be written as an extension, using custom functions as the 
> API, instead of extra SQL syntax.
> 
> 2. It's not very useful if it's just a dummy replacement for "WHERE 
> random() < ?". It has to be more advanced than that. Quality of the 
> sample is important, as is performance. There was also an interesting 
> idea of on implementing monetary unit sampling.
> 
> I think this would be a useful project if those two points are taken 
> care of.
> 
> Another idea that Robert Haas suggested was to add support doing a TID 
> scan for a query like "WHERE ctid<  '(501,1)'". That's not enough work 
> for GSoC project on its own, but could certainly be a part of it.
> 
> -- 
>    Heikki Linnakangas
>    EnterpriseDB   http://www.enterprisedb.com
> 
> -- 
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

 		 	   		  

In response to

Responses

pgsql-hackers by date

Next:From: Qi HuangDate: 2012-04-17 12:14:20
Subject: Re: Gsoc2012 idea, tablesample
Previous:From: Thom BrownDate: 2012-04-17 11:10:01
Subject: Re: Slow temporary tables when using sync rep

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group