Re: [HACKERS] Parallel Hash take II

From: Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Rushabh Lathia <rushabh(dot)lathia(at)gmail(dot)com>, Prabhat Sahu <prabhat(dot)sahu(at)enterprisedb(dot)com>, Peter Geoghegan <pg(at)bowt(dot)ie>, Robert Haas <robertmhaas(at)gmail(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Rafia Sabih <rafia(dot)sabih(at)enterprisedb(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>, Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>, Oleg Golovanov <rentech(at)mail(dot)ru>
Subject: Re: [HACKERS] Parallel Hash take II
Date: 2017-11-27 09:25:12
Message-ID: CAEepm=0oE=yO0Kam86W1d-iJoasWByYkcrkDoJu6t5mRhFGHkQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

On Thu, Nov 23, 2017 at 12:36 AM, Thomas Munro
<thomas(dot)munro(at)enterprisedb(dot)com> wrote:
> Here's a new patch set with responses to the last batch of review comments.

Rebased on top of the recent SGML->XML change. Also:

1. The final patch in the v26 patchset extended EXPLAIN ANALYZE
output to show per-worker information. I'm withdrawing that patch for
now. If you want to see how many tuples each backend hashed you can
already do that with (ANALYZE, VERBOSE). It's a pre-existing bug that
you don't get batch/bucket/size info when Hash Join doesn't run in the
leader, and it's a pre-existing bug that EXPLAIN doesn't show
information for the leader separately. I decided that it's not this
patchset's job to fix that stuff, and it's not entirely clear what the
best approach is anyway. Let's discuss the way that information is
captured and displayed separately from the Parallel Hash feature.

2. I found a way to crash v26 by starting a worker very late. Fixed.

Unfortunately I saw a one-off case of an assertion failure in
ExecParallelHashRepartitionRest()/sts_begin_parallel_scan() on Travis
CI that I can't explain. I haven't been able to reproduce it there or
on any other machine since. I am still looking into it.

--
Thomas Munro
http://www.enterprisedb.com

Attachment Content-Type Size
parallel-hash-v27.patchset.tgz application/x-gzip 57.8 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jakub Glapa 2017-11-27 09:54:03 ERROR: too many dynamic shared memory segments
Previous Message hubert depesz lubaczewski 2017-11-27 09:15:08 Re: explain analyze output with parallel workers - question about meaning of information for explain.depesz.com