Re: pgsql: Support parallel btree index builds.

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Andres Freund <andres(at)anarazel(dot)de>, Peter Geoghegan <pg(at)bowt(dot)ie>, Rushabh Lathia <rushabh(dot)lathia(at)gmail(dot)com>, pgsql-committers <pgsql-committers(at)postgresql(dot)org>
Subject: Re: pgsql: Support parallel btree index builds.
Date: 2018-02-06 16:05:21
Message-ID: 15462.1517933121@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers pgsql-hackers

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> On Tue, Feb 6, 2018 at 10:46 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> ... I, at least, don't have that understanding from looking
>> at the thread. For one thing, Peter has not explained why this issue
>> appears now with parallel index build when it did not before; it's
>> not like logtape.c isn't old enough to vote.

> Yeah, he has actually. In other cases, the buffer is guaranteed to
> have been filled at least once (and thus, from valgrind's point of
> view, is initialized) because if that weren't going to happen then we
> would have not have switched to a tape-sort in the first place. You
> can't set work_mem smaller than 8kB. But in the parallel case each
> worker must always produce a tape, so it can happen if a worker is
> unlucky enough to get only a very small slice of the data (because the
> other participants gobble it all up before that process really gets
> going).

Ah, I see. So this is really a problem that's been latent all along,
but was never exposed in any previous use-case for logtape.c.

> But what I really need here is
> some input on an option you do like, not just a list of things you
> don't like.

I like the option of doing VALGRIND_MAKE_MEM_DEFINED on the tail
portion of the buffer before writing it. That seems pretty tightly
tied to the behavior we're decreeing valid, whereas the suppression
is not.

regards, tom lane

In response to

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Peter Geoghegan 2018-02-06 16:32:08 Re: pgsql: Support parallel btree index builds.
Previous Message Robert Haas 2018-02-06 15:59:56 Re: pgsql: Support parallel btree index builds.

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2018-02-06 16:12:37 Re: [HACKERS] [PATCH] Lockable views
Previous Message Robert Haas 2018-02-06 15:59:56 Re: pgsql: Support parallel btree index builds.