Re: Making "COPY partitioned_table FROM" faster

On 24 July 2018 at 06:38, Peter Eisentraut
<peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:
> On 20.07.18 16:57, David Rowley wrote:
>> One final note: I'm not entirely convinced we need this adaptive
>> code, but it seems easy enough to rip it back out if it's more trouble
>> than it's worth. But if the other option is a GUC, then I'd rather
>> stick with the adaptive code, it's likely going to do much better than
>> a GUC since it can change itself during the copy which will be useful
>> when the stream contains a mix small and large sets of consecutive
>> tuples which belong to the same partition.
>
> I think some kind of way to switch between the two code paths would be
> desirable. For example, with hash partitioning, it's likely that in
> many cases you won't find any adjacent candidates in batches of
> significant size. So then you've just made everything 5% slower.
> Unless we can make the multi-insert path itself faster.
>
> The particular heuristic you have chosen seems sensible to me, but we'll
> have to see how it holds up in practice.

Thanks for having a look at this. You're probably right here,
although, the slowdown I measured was 2.2%, not 5%. I had it in my
head that it was about 1%, but for 2.2% it seems worth the extra code.

I've attached a small delta against the v2 patch that fixes up a few
comments and gets rid of a needless assignment.

--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services