Re: pg_dump test instability

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_dump test instability
Date: 2018-09-13 21:03:39
Message-ID: 6727.1536872619@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I wrote:
> Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> writes:
>> I see. Why not shift all items up to the i'th up by one place, instead
>> of moving only the first one? That way the sortedness would be
>> preserved. Otherwise we'd move the first one into the middle, then
>> sorting would move it to the front again, etc.

> Hmmm ... might be worth doing, but I'm not sure. The steady-state cycle
> will probably be that after one task has been dispatched, we'll sleep
> until some task finishes and then that will unblock some pending items,
> resulting in new entries at the end of the list, forcing a sort anyway
> before we next dispatch a task. So I was expecting that avoiding a sort
> here wasn't really going to be worth expending much effort for. But my
> intuition about that could be wrong. I'll run a test case with some
> instrumentation added and see how often we could avoid sorts by
> memmove'ing.

OK, my intuition was faulty. At least when testing with the regression
database, situations where we are taking the slow path at all seem to
involve several interrelated dump objects (eg indexes of a table) that
are all waiting for the same lock, such that we may be able to dispatch a
number of unrelated tasks before anything gets added from the pending
list. Doing it as you suggest eliminates a significant fraction of
the re-sort operations.

Attached updated patch does it like that and makes the cosmetic
adjustments you suggested. I also went ahead and did the renaming
of par_prev/par_next/par_list_xxx that I'd suggested upthread.
I think this is committable ...

regards, tom lane

Attachment Content-Type Size
smarter-parallel-dump-restore-2.patch text/x-diff 51.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Gierth 2018-09-13 21:12:03 Re: Consistent segfault in complex query
Previous Message Jerry Jelinek 2018-09-13 20:56:42 Re: patch to allow disable of WAL recycling