From:
Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
To:
Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
Cc:
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, David Fetter <david(at)fetter(dot)org>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject:
Re: wip: functions median and percentile
Date:
2010-10-01 19:17:08
Message-ID:
AANLkTinZwWHnssyER3Hi+L=V42O9sF43r_Duw7UL9H=6@mail.gmail.com (view raw or flat )
Thread:
2010-08-19 10:59:33 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-08-19 14:50:23 from Greg Stark <gsstark(at)mit(dot)edu>
2010-08-19 15:33:13 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-08-19 16:49:45 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-08-19 17:03:20 from David Fetter <david(at)fetter(dot)org>
2010-08-19 17:12:12 from "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
2010-08-19 17:14:58 from David Fetter <david(at)fetter(dot)org>
2010-08-19 17:25:36 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-08-19 17:47:59 from David Fetter <david(at)fetter(dot)org>
2010-08-19 18:17:42 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-08-19 17:41:57 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-08-19 18:17:28 from "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
2010-08-19 17:38:14 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-09-20 03:47:09 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-09-20 05:11:02 from David Fetter <david(at)fetter(dot)org>
2010-09-20 06:22:29 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-09-21 20:28:15 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-09-21 20:44:16 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-09-21 21:03:31 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-09-22 02:21:01 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-09-22 04:36:24 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-09-22 03:06:51 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-09-22 04:44:58 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-09-23 09:35:47 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-09-23 17:45:36 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-09-23 18:27:38 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-09-23 18:48:47 from David Fetter <david(at)fetter(dot)org>
2010-09-23 18:56:53 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-09-26 10:39:21 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-01 08:17:19 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-10-01 08:46:45 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-01 13:05:43 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-10-01 13:19:57 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-01 14:11:08 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-10-01 14:43:47 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-10-01 15:00:35 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-10-01 15:08:00 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-10-01 15:15:03 from "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
2010-10-01 15:35:03 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-10-01 15:16:03 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-10-01 15:22:46 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-10-01 15:32:03 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-10-01 17:37:03 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-01 19:17:08 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-03 14:06:25 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-10-03 15:49:37 from Greg Stark <gsstark(at)mit(dot)edu>
2010-10-04 06:36:48 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-10-04 06:58:44 from Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
2010-10-04 16:14:06 from Greg Stark <gsstark(at)mit(dot)edu>
2010-10-04 17:22:39 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-10-04 19:20:28 from Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
2010-10-05 06:04:31 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-10-05 08:47:52 from Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
2010-10-05 12:14:24 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-10-05 13:08:59 from Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
2010-10-10 21:16:59 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-10-11 06:58:53 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-11 09:35:13 from Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
2010-10-11 09:55:16 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-11 10:33:11 from Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
2010-10-11 14:03:40 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-10-11 15:30:46 from Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
2010-10-11 15:44:08 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-10-11 17:34:51 from Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
2010-10-11 17:37:11 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-10-11 17:42:10 from Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
2010-10-11 17:48:12 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-10-11 17:55:32 from Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
2010-10-11 18:05:11 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-10-11 18:23:25 from Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
2010-10-11 10:33:12 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-10-11 14:08:53 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-10-11 14:29:38 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-10-11 18:27:13 from Greg Stark <gsstark(at)mit(dot)edu>
2010-10-11 18:30:39 from Josh Berkus <josh(at)agliodbs(dot)com>
2010-10-11 18:46:47 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-12 03:08:32 from Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
2010-10-12 08:44:20 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-13 10:36:59 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-13 10:48:17 from Peter Eisentraut <peter_e(at)gmx(dot)net>
2010-10-13 10:51:51 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-13 10:56:06 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-14 00:54:42 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-10-14 02:37:37 from Greg Stark <gsstark(at)mit(dot)edu>
2010-10-14 12:17:54 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-10-14 04:53:57 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-03 17:31:18 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-10-01 14:44:44 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-09-23 13:22:39 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2010-09-23 13:25:32 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
Lists:
pgsql-hackers pgsql-rrreviewers
Hello
updated version
* memsort removed
* window aggregate support blocked
Regards
Pavel
2010/10/1 Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>:
> 2010/10/1 Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>:
>> 2010/10/2 Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>:
>>> Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com> writes:
>>>> 2010/10/2 Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>:
>>>>> The implementation I would've expected to see is to do the sort and then
>>>>> have two code paths for retrieving the median, depending on whether the
>>>>> sort result is all in memory or not.
>>>
>>>> Hm? The problem we encountered in the middle of the patch is there is
>>>> no chance to call tuplesort_end if median is called in moving frame
>>>> window aggregate because final function is called multiple times
>>>> during moving.
>>>
>>> Well, if you haven't got a solution for that, then this patch isn't
>>> ready for prime time.
>>>
>>> It's entirely possible that median as a window function is intractable.
>>> I'd rather have it throwing error than offer an implementation that will
>>> fall over as soon as the window gets large.
>>
>> Well, that sounds like the conclusion. It is a shame, but we have to
>> throw an error from median() in the window aggregate, if Pavel does
>> not have any better solution. And as an aggregate function only, the
>> patch is ready if the window-related parts are removed.
>>
>
> I am sorry - I don't have a better solution. Classic algorithm isn't
> well for window aggregate - it needs a sort after any append a new
> item. Maybe we can use a separate functionality based on estimated
> values for a windows. I read some articles about it. But this is work
> on longer time - all articles about this topic are experimental. More
> I am not mathematician - so I am not able to review these methods.
> Today or tomorrow I'll send a updated patch without support a window
> aggregates.
>
> Regards
>
> Pavel Stehule
>
>> Regards,
>>
>>
>> --
>> Hitoshi Harada
>>
>
Attachment: median.diff
Description: application/octet-stream (16.2 KB)
In response to
Responses
pgsql-hackers by date
Next :From: Tom LaneDate: 2010-10-01 19:25:08
Subject : Re: timestamp_in DirectFunctionCall
Previous :From : Tom LaneDate : 2010-10-01 19:14:14
Subject : Re: So git pull is shorthand for what exactly?
pgsql-rrreviewers by date
Next :From: Jaime CasanovaDate: 2010-10-01 20:42:54
Subject : Re: Patch reviewers
Previous :From : ๏̯͡๏ Guido BarosioDate : 2010-10-01 17:48:11
Subject : Patch reviewers