Re: Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 2:46 PM, Peter Geoghegan <pg(at)heroku(dot)com> wrote:
> Feel free to make a counter-proposal for a cap. I'm not attached to
> 500. I'm mostly worried about blatant waste with very large workMem
> sizings. Tens of thousands of tapes is just crazy. The amount of data
> that you need to have as input is very large when workMem is big
> enough for this new cap to be enforced.

If tuplesort callers passed a hint about the number of tuples that
would ultimately be sorted, and (for the sake of argument) it was
magically 100% accurate, then theoretically we could just allocate the
right number of tapes up-front. That discussion is a big can of worms,
though. There are of course obvious disadvantages that come with a
localized cost model, even if you're prepared to add some "slop" to
the allocation size or whatever.

--
Peter Geoghegan