Re: Parallel query execution

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Claudio Freire <klaussfreire(at)gmail(dot)com>
Cc: Jeff Janes <jeff(dot)janes(at)gmail(dot)com>, Stephen Frost <sfrost(at)snowman(dot)net>, Gavin Flower <GavinFlower(at)archidevsys(dot)co(dot)nz>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Parallel query execution
Date: 2013-01-17 03:29:24
Message-ID: 20130117032924.GC683@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Jan 16, 2013 at 11:56:21PM -0300, Claudio Freire wrote:
> On Wed, Jan 16, 2013 at 11:44 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > On Wed, Jan 16, 2013 at 05:04:05PM -0800, Jeff Janes wrote:
> >> On Tuesday, January 15, 2013, Stephen Frost wrote:
> >>
> >> * Gavin Flower (GavinFlower(at)archidevsys(dot)co(dot)nz) wrote:
> >> > How about being aware of multiple spindles - so if the requested
> >> > data covers multiple spindles, then data could be extracted in
> >> > parallel. This may, or may not, involve multiple I/O channels?
> >>
> >> Yes, this should dovetail with partitioning and tablespaces to pick up
> >> on exactly that.
> >>
> >>
> >> I'd rather not have the benefits of parallelism be tied to partitioning if we
> >> can help it. Hopefully implementing parallelism in core would result in
> >> something more transparent than that.
> >
> > We will need a way to know we are not saturating the I/O channel with
> > random I/O that could have been sequential if it was single-threaded.
> > Tablespaces give us that info; not sure what else does.
>
> I do also think tablespaces are a safe bet. But it wouldn't help for
> parallelizing sorts or other operations with tempfiles (tempfiles
> reside on the same tablespace), or even over a single table (same

We can round-robin temp tablespace usage if you list multiple entries.

> tablespace again). And when the query is CPU-bound, it could be
> parallelized by simply making a multithreaded memory sort. Well, not
> so simply, but I do think it's an important building block.

Yes, and detecting when to use these parallel features will be hard.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Janes 2013-01-17 03:32:28 Re: Parallel query execution
Previous Message Noah Misch 2013-01-17 03:26:01 Re: pgcrypto seeding problem when ssl=on