From:
Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
To:
Stephen Frost <sfrost(at)snowman(dot)net>
Cc:
Bruce Momjian <bruce(at)momjian(dot)us>, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>, Josh Berkus <josh(at)agliodbs(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject:
Re: Parallel query execution
Date:
2013-01-17 03:32:28
Message-ID:
CAMkU=1zU9myesU-z-F+a1MZC22USAirK=gyJuFJ3_HDN+h9BhQ@mail.gmail.com (view raw or flat )
Thread:
2013-01-15 22:14:19 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-15 22:33:16 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-15 22:39:10 from Peter Geoghegan <peter(at)2ndquadrant(dot)com>
2013-01-15 22:45:15 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-15 22:53:29 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2013-01-15 22:55:22 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-15 23:01:04 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2013-01-15 23:03:33 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-16 11:52:43 from Magnus Hagander <magnus(at)hagander(dot)net>
2013-01-16 13:11:06 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-16 17:16:27 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-16 17:50:07 from Josh Berkus <josh(at)agliodbs(dot)com>
2013-01-16 17:56:01 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-16 18:00:22 from Josh Berkus <josh(at)agliodbs(dot)com>
2013-01-16 05:35:11 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2013-01-15 23:03:50 from Gavin Flower <GavinFlower(at)archidevsys(dot)co(dot)nz>
2013-01-15 23:08:47 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-15 23:15:57 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-15 23:19:27 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-16 02:36:55 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2013-01-16 03:13:33 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-16 03:16:56 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2013-01-16 03:55:13 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-16 04:47:21 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2013-01-16 13:33:54 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-16 15:23:14 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2013-01-16 03:56:52 from Josh Berkus <josh(at)agliodbs(dot)com>
2013-01-16 03:59:35 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-16 04:22:42 from Josh Berkus <josh(at)agliodbs(dot)com>
2013-01-16 04:28:18 from Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
2013-01-16 04:32:54 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-16 04:37:28 from Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
2013-01-16 17:11:42 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-16 20:07:14 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-16 21:06:51 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2013-01-16 21:18:38 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-16 21:57:01 from "Dickson S(dot) Guedes" <listas(at)guedesoft(dot)net>
2013-01-16 22:00:54 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-17 03:32:28 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2013-01-16 04:48:29 from Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
2013-01-16 07:07:29 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-16 08:21:18 from Abhijit Menon-Sen <ams(at)2ndQuadrant(dot)com>
2013-01-16 12:08:27 from Magnus Hagander <magnus(at)hagander(dot)net>
2013-01-16 13:18:17 from Abhijit Menon-Sen <ams(at)2ndQuadrant(dot)com>
2013-01-16 13:43:18 from Boszormenyi Zoltan <zb(at)cybertec(dot)at>
2013-01-21 23:23:07 from Phil Sorber <phil(at)omniti(dot)com>
2013-01-22 00:31:12 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-22 07:29:29 from Magnus Hagander <magnus(at)hagander(dot)net>
2013-01-16 15:56:14 from Noah Misch <noah(at)leadboat(dot)com>
2013-01-16 12:12:00 from Simon Riggs <simon(at)2ndquadrant(dot)com>
2013-01-16 12:18:04 from Magnus Hagander <magnus(at)hagander(dot)net>
2013-01-16 13:58:12 from Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
2013-01-16 14:02:45 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-16 14:10:54 from Abhijit Menon-Sen <ams(at)2ndQuadrant(dot)com>
2013-01-16 14:17:54 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-16 19:28:05 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-16 19:55:15 from Josh Berkus <josh(at)agliodbs(dot)com>
2013-01-16 20:13:50 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-16 20:45:46 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-17 05:01:26 from Peter Eisentraut <peter_e(at)gmx(dot)net>
2013-01-17 06:06:13 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-17 01:34:41 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2013-01-17 03:40:07 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-17 04:48:59 from Abhijit Menon-Sen <ams(at)2ndQuadrant(dot)com>
2013-01-17 06:52:53 from Craig Ringer <craig(at)2ndQuadrant(dot)com>
2013-01-17 07:05:05 from Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
2013-01-17 07:15:52 from Abhijit Menon-Sen <ams(at)2ndQuadrant(dot)com>
2013-01-17 08:43:51 from Magnus Hagander <magnus(at)hagander(dot)net>
2013-01-17 13:17:07 from Craig Ringer <craig(at)2ndQuadrant(dot)com>
2013-01-17 14:54:43 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-16 14:15:44 from Craig Ringer <craig(at)2ndQuadrant(dot)com>
2013-01-16 14:16:33 from Craig Ringer <craig(at)2ndQuadrant(dot)com>
2013-01-16 17:36:10 from Josh Berkus <josh(at)agliodbs(dot)com>
2013-01-16 22:01:12 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-17 04:38:51 from Craig Ringer <craig(at)2ndQuadrant(dot)com>
2013-01-17 07:19:08 from Pavan Deolasee <pavan(dot)deolasee(at)gmail(dot)com>
2013-01-17 09:22:48 from Magnus Hagander <magnus(at)hagander(dot)net>
2013-01-19 22:21:29 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2013-01-20 18:42:45 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-20 19:39:05 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2013-01-20 20:18:34 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2013-01-20 21:57:57 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-21 00:07:21 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2013-01-21 02:37:32 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-21 03:04:22 from Andrew Dunstan <andrew(at)dunslane(dot)net>
2013-01-21 03:07:07 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-21 03:11:34 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-21 22:48:38 from Dimitri Fontaine <dimitri(at)2ndQuadrant(dot)fr>
2013-01-22 01:23:00 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-22 01:47:12 from Josh Berkus <josh(at)agliodbs(dot)com>
2013-01-22 02:27:55 from Phil Sorber <phil(at)omniti(dot)com>
2013-01-22 05:24:18 from Pavan Deolasee <pavan(dot)deolasee(at)gmail(dot)com>
2013-01-22 06:15:38 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-22 09:35:51 from Dimitri Fontaine <dimitri(at)2ndQuadrant(dot)fr>
2013-01-22 17:09:02 from Gavin Flower <GavinFlower(at)archidevsys(dot)co(dot)nz>
2013-01-22 12:44:41 from Pavan Deolasee <pavan(dot)deolasee(at)gmail(dot)com>
2013-01-22 13:16:40 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2013-01-22 14:22:04 from Andrew Dunstan <andrew(at)dunslane(dot)net>
2013-01-23 16:44:29 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-23 17:08:39 from Andres Freund <andres(at)2ndquadrant(dot)com>
2013-01-23 17:51:15 from Josh Berkus <josh(at)agliodbs(dot)com>
2013-01-23 18:27:14 from "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
2013-01-23 18:44:03 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-23 20:23:32 from Phil Sorber <phil(at)omniti(dot)com>
2013-01-23 20:31:00 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-23 20:48:50 from Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>
2013-01-23 21:27:58 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-23 21:47:58 from Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
2013-01-24 11:33:37 from Amit Kapila <amit(dot)kapila(at)huawei(dot)com>
2013-01-24 09:13:39 from Pavan Deolasee <pavan(dot)deolasee(at)gmail(dot)com>
2013-01-22 03:33:49 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-22 02:22:47 from Phil Sorber <phil(at)omniti(dot)com>
2013-01-22 13:15:28 from Dimitri Fontaine <dimitri(at)2ndQuadrant(dot)fr>
2013-01-21 07:48:45 from Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>
2013-01-21 18:54:33 from Josh Berkus <josh(at)agliodbs(dot)com>
2013-01-21 19:04:14 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2013-01-21 19:09:57 from Josh Berkus <josh(at)agliodbs(dot)com>
2013-01-21 19:39:20 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-23 17:19:08 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-16 09:37:04 from Daniel Farina <daniel(at)heroku(dot)com>
2013-01-16 13:42:29 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-16 16:02:10 from Noah Misch <noah(at)leadboat(dot)com>
2013-01-16 12:05:36 from Andres Freund <andres(at)2ndquadrant(dot)com>
2013-01-16 12:53:12 from Robert Haas <robertmhaas(at)gmail(dot)com>
2013-01-16 13:38:08 from Stephen Frost <sfrost(at)snowman(dot)net>
2013-01-16 17:13:51 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-16 14:05:39 from Andrew Dunstan <andrew(at)dunslane(dot)net>
2013-01-16 17:20:24 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-16 17:29:16 from Andrew Dunstan <andrew(at)dunslane(dot)net>
2013-01-17 01:04:05 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2013-01-17 02:44:53 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-17 02:56:21 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2013-01-17 03:29:24 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-17 01:04:05 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2013-01-17 02:42:04 from Claudio Freire <klaussfreire(at)gmail(dot)com>
2013-01-16 00:11:20 from Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
2013-01-16 02:29:01 from Bruce Momjian <bruce(at)momjian(dot)us>
2013-01-24 22:34:49 from Paul Ramsey <pramsey(at)cleverelephant(dot)ca>
2013-01-24 23:17:18 from Bruce Momjian <bruce(at)momjian(dot)us>
Lists:
pgsql-hackers
On Wednesday, January 16, 2013, Stephen Frost wrote:
> * Bruce Momjian (bruce(at)momjian(dot)us <javascript:;>) wrote:
> > I am not sure how a COPY could be easily parallelized, but I supposed it
> > could be done as part of the 1GB segment feature. People have
> > complained that COPY is CPU-bound, so it might be very interesting to
> > see if we could offload some of that parsing overhead to a child.
>
> COPY can certainly be CPU bound but before we can parallelize that
> usefully we need to solve the problem around extent locking when trying
> to do multiple COPY's to the same table.
>
I think that is rather over-stating it. Even with unindexed untriggered
tables, I can get some benefit from doing hand-rolled parallel COPY before
the extension lock becomes an issue, at least on some machines. And with
triggered or indexed tables, all the more so.
Cheers,
Jeff
In response to
pgsql-hackers by date
Next :From: Tom LaneDate: 2013-01-17 03:40:07Subject : Re: CF3+4Previous :From : Bruce MomjianDate : 2013-01-17 03:29:24Subject : Re: Parallel query execution
