Re: Tuplesort merge pre-reading

On Sat, Sep 17, 2016 at 9:41 AM, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
> Ok. I'll probably read through it myself once more some time next week, and
> also have a first look at your actual parallel sorting patch. Have a good
> trip!

Thanks! It will be good to get away for a while.

I'd be delighted to recruit you to work on the parallel CREATE INDEX
patch. I've already explained how I think this preread patch of yours
works well with parallel tuplesort (as proposed) in particular. To
reiterate: while what you've come up with here is technically an
independent improvement to merging, it's much more valuable in the
overall context of parallel sort, where disproportionate wall clock
time is spent merging, and where multiple passes are the norm (one
pass in each worker, plus one big final pass in the leader process
alone -- logtape.c fragmentation becomes a real issue). The situation
is actually similar to the original batch memory preloading patch that
went into 9.6 (which your new patch supersedes), and the subsequently
committed quicksort for external sort patch (which my new patch
extends to work in parallel).

Because I think of your preload patch as a part of the overall
parallel CREATE INDEX effort, if that was the limit of your
involvement then I'd still think it fair to credit you as my
co-author. I hope it isn't the limit of your involvement, though,
because it seems likely that the final result will be better still if
you get involved with the big patch that formally introduces parallel
CREATE INDEX.

--
Peter Geoghegan