Re: Parallel query execution

From: Claudio Freire <klaussfreire(at)gmail(dot)com>
To: Stephen Frost <sfrost(at)snowman(dot)net>
Cc: Bruce Momjian <bruce(at)momjian(dot)us>, Gavin Flower <GavinFlower(at)archidevsys(dot)co(dot)nz>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Parallel query execution
Date: 2013-01-16 03:16:56
Message-ID: CAGTBQpZwjc4wvr+fm_zE1=Oi0iVYPM35CaN1MOqmfVEy3omGug@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Jan 16, 2013 at 12:13 AM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> * Claudio Freire (klaussfreire(at)gmail(dot)com) wrote:
>> On Tue, Jan 15, 2013 at 8:19 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> > The 1GB idea is interesting. I found in pg_upgrade that file copy would
>> > just overwhelm the I/O channel, and that doing multiple copies on the
>> > same device had no win, but those were pure I/O operations --- a
>> > sequential scan might be enough of a mix of I/O and CPU that parallelism
>> > might help.
>>
>> AFAIR, synchroscans were introduced because multiple large sequential
>> scans were counterproductive (big time).
>
> Sequentially scanning the *same* data over and over is certainly
> counterprouctive. Synchroscans fixed that, yes. That's not what we're
> talking about though- we're talking about scanning and processing
> independent sets of data using multiple processes.

I don't see the difference. Blocks are blocks (unless they're cached).

> It's certainly
> possible that in some cases that won't be as good

If memory serves me correctly (and it does, I suffered it a lot), the
performance hit is quite considerable. Enough to make it "a lot worse"
rather than "not as good".

> but there will be
> quite a few cases where it's much, much better.

Just cached segments.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Stephen Frost 2013-01-16 03:47:57 Re: log_lock_waits to identify transaction's relation
Previous Message Stephen Frost 2013-01-16 03:13:33 Re: Parallel query execution