Re: Parallel query execution

From: Claudio Freire <klaussfreire(at)gmail(dot)com>
To: Bruce Momjian <bruce(at)momjian(dot)us>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, Gavin Flower <GavinFlower(at)archidevsys(dot)co(dot)nz>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Parallel query execution
Date: 2013-01-16 02:36:55
Message-ID: CAGTBQpbtQE2PgfORWgLez5+1p4Ogo+TF3_x-fA91QqHp=adAGA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Jan 15, 2013 at 8:19 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> Given our row-based storage architecture, I can't imagine we'd do
>> anything other than take a row-based approach to this.. I would think
>> we'd do two things: parallelize based on partitioning, and parallelize
>> seqscan's across the individual heap files which are split on a per-1G
>> boundary already. Perhaps we can generalize that and scale it based on
>> the number of available processors and the size of the relation but I
>> could see advantages in matching up with what the kernel thinks are
>> independent files.
>
> The 1GB idea is interesting. I found in pg_upgrade that file copy would
> just overwhelm the I/O channel, and that doing multiple copies on the
> same device had no win, but those were pure I/O operations --- a
> sequential scan might be enough of a mix of I/O and CPU that parallelism
> might help.

AFAIR, synchroscans were introduced because multiple large sequential
scans were counterproductive (big time).

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Simon Riggs 2013-01-16 02:40:32 Sequence Access Method WIP
Previous Message Bruce Momjian 2013-01-16 02:29:01 Re: Parallel query execution