Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

From: Peter Geoghegan <pg(at)bowt(dot)ie>
To: Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Rushabh Lathia <rushabh(dot)lathia(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Corey Huinker <corey(dot)huinker(at)gmail(dot)com>
Subject: Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)
Date: 2018-01-25 02:05:46
Message-ID: CAH2-Wz=Fpqg4wmcM6dE92CawyRrVvAG_ioP2DbnBSpMAcOaDEA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Jan 24, 2018 at 5:31 PM, Thomas Munro
<thomas(dot)munro(at)enterprisedb(dot)com> wrote:
> Here's a version that works, and a minimal repro test module thing.
> Without 0003 applied, it hangs.

I can confirm that this version does in fact fix the problem with
parallel CREATE INDEX hanging in the event of (simulated) worker
fork() failure. And, it seems to have at least one tiny advantage over
the other approaches I was talking about that you didn't mention,
which is that we never have to wait until the leader stops
participating as a worker before an error is raised. IOW, either the
whole parallel CREATE INDEX operation throws an error at an early
point in the CREATE INDEX, or the CREATE INDEX completely succeeds.

Obviously, the other, stated advantage is more relevant: *everyone*
automatically doesn't have to worry about nworkers_launched being
inaccurate this way, including code that gets away with this today
only due to using a tuple queue, such as nodeGather.c, but may not
always get away with it in the future.

I've run out of time to assess what you've done here in any real
depth. For now, I will say that this approach seems interesting to me.
I'll take a closer look tomorrow.

--
Peter Geoghegan

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Langote 2018-01-25 02:11:57 Re: non-bulk inserts and tuple routing
Previous Message Masahiko Sawada 2018-01-25 01:41:55 Re: Failed to request an autovacuum work-item in silence