From:
Greg Stark <stark(at)mit(dot)edu>
To:
Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
Cc:
Simon Riggs <simon(at)2ndquadrant(dot)com>, Peter Geoghegan <peter(at)2ndquadrant(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PG Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject:
Re: Inlining comparators as a performance optimisation
Date:
2011-09-21 12:47:46
Message-ID:
CAM-w4HOGCv3J=JONM6m4c0rnCmKsFY=dogFd2P60=AAifBGWsQ@mail.gmail.com (view raw or flat )
Thread:
2011-09-20 01:56:01 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-09-20 02:51:32 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-09-20 23:53:16 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-09-21 01:54:20 from Dan McGee <dan(at)archlinux(dot)org>
2011-09-21 06:51:32 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2011-09-21 07:01:11 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2011-09-21 07:08:42 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2011-09-21 12:47:46 from Greg Stark <stark(at)mit(dot)edu>
2011-09-21 13:34:49 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-09-21 15:22:50 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-09-21 19:21:38 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2011-09-21 14:20:55 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-09-21 14:31:11 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2011-09-21 14:50:22 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-09-21 15:03:15 from Andrew Dunstan <andrew(at)dunslane(dot)net>
2011-09-21 15:46:52 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-09-21 15:56:56 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2011-09-21 16:13:07 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2012-01-13 09:48:56 from "Pierre C" <lists(at)peufeu(dot)com>
2012-01-28 14:55:04 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-09-21 16:04:21 from Greg Stark <stark(at)mit(dot)edu>
2011-09-21 16:23:13 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-09-21 16:35:41 from Greg Stark <stark(at)mit(dot)edu>
2011-09-26 02:12:55 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-09-26 03:46:29 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-09-26 13:53:49 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-10-05 01:49:40 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-10-05 02:55:28 from Greg Stark <stark(at)mit(dot)edu>
2011-10-05 13:55:04 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-10-05 15:31:57 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-11-18 05:20:26 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-11-18 08:53:09 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2011-11-18 13:39:38 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-11-18 14:11:27 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-11-23 14:48:22 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2011-11-18 21:38:18 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-11-19 02:55:54 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-11-20 03:29:07 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-11-20 04:13:36 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-11-21 02:02:06 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-11-18 22:08:18 from Martijn van Oosterhout <kleptog(at)svana(dot)org>
2011-09-21 16:13:20 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2011-09-21 16:27:49 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-09-21 15:43:31 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-11-21 22:55:18 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-11-23 01:09:23 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-11-23 19:24:22 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-11-25 01:05:15 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-11-26 18:58:00 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-11-28 02:15:13 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-11-29 15:31:45 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-11-29 18:48:37 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-11-29 20:34:40 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-11-29 21:36:26 from Andres Freund <andres(at)anarazel(dot)de>
2011-11-29 21:43:40 from Greg Jaskiewicz <gj(at)pointblue(dot)com(dot)pl>
2011-12-01 16:44:55 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-12-01 17:15:52 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-02 01:13:02 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-02 02:15:14 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-02 02:46:09 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-02 03:22:32 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-02 03:48:07 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-02 04:00:36 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-02 05:16:08 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-02 13:57:48 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-02 15:11:19 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-02 19:35:32 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-12-02 20:04:45 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2011-12-02 20:21:50 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-02 20:42:21 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-12-03 18:36:04 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-04 19:17:15 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-05 00:14:33 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-12-05 13:23:45 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2011-12-05 17:10:05 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-12-05 17:53:23 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-06 10:40:23 from Nicolas Barbier <nicolas(dot)barbier(at)gmail(dot)com>
2011-12-06 16:49:03 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-06 17:06:09 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-06 17:48:35 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-06 18:07:08 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-06 18:08:29 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-06 21:23:17 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-07 00:18:24 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-07 01:13:59 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-12-07 03:45:20 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-07 11:57:08 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-12-07 01:46:27 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-07 14:39:54 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-07 15:09:32 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-12-07 15:15:35 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-07 15:58:03 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-12-07 16:22:30 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-12-16 19:55:12 from Greg Smith <greg(at)2ndQuadrant(dot)com>
2011-12-02 13:33:30 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-12-02 01:29:56 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2011-12-02 01:31:06 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-09-21 06:55:55 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2011-09-20 13:57:24 from Amit Kapila <amit(dot)kapila(at)huawei(dot)com>
Lists:
pgsql-hackers
On Wed, Sep 21, 2011 at 8:08 AM, Heikki Linnakangas
<heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>> How about almost every primary index creation?
>
> Nope. Swamped by everything else.
Really? I think it's pretty common for shops to be able to dedicate
large amounts of RAM to building initial indexes on data loads or
reindex operations. Enough that they can cache the entire table for
the short time they're doing the index builds even if they're quite
large. Witness the recent pleas to allow maintenance_work_mem on the
order of tens of gigabytes. And it's also pretty common that shops can
dedicate very large I/O bandwidth, in many cases enough to saturate
the memory bandwidth, for doing these kinds of batch operations when
they get large enough to need to do an external sort.
There's still overhead of reading the pages, the tuples, finding the
sort keys in the tuple, etc. But I think the actual qsort or heap
operations in tapesort are pretty big portions of the work.
This is pretty easy to measure. Just run oprofile or gprof and see
what percentage of time for a big index build is spent in qsort.
--
greg
In response to
Responses
pgsql-hackers by date
Next :From: Robert HaasDate: 2011-09-21 13:34:49Subject : Re: Inlining comparators as a performance optimisationPrevious :From : Robert HaasDate : 2011-09-21 12:47:00Subject : Re: Range Types - typo + NULL string constructor
