Re: [HACKERS] [PATCH] Incremental sort

Hi!

Thank you for reviewing this patch!
Revised version is attached.

On Mon, Mar 5, 2018 at 1:19 PM, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
wrote:

> I have started reviewing the patch and doing some testing, and I have
> pretty quickly ran into a segfault. Attached is a simple reproducer and
> an backtrace. AFAICS the bug seems to be somewhere in the tuplesort
> changes, likely resetting a memory context too soon or something like
> that. I haven't investigated it further, but it matches my hunch that
> tuplesort is likely where the bugs will be.
>

Right. Incremental sort patch introduces maincontext of memory which
is persistent between incremental sort groups. But mergeruns()
reallocates memtuples in sortcontext which is cleared by tuplesort_reset().
Fixed in the revised patch.

Otherwise the patch seems fairly complete. A couple of minor things that
> I noticed while eyeballing the changes in a diff editor.
>
>
> 1) On a couple of places the new code has this comment
>
> /* even when not parallel-aware */
>
> while all the immediately preceding blocks use
>
> /* even when not parallel-aware, for EXPLAIN ANALYZE */
>
> I suggest using the same comment, otherwise it kinda suggests it's not
> because of EXPLAIN ANALYZE.
>

Right, fixed. I also found that incremental sort shoudn't support
DSM reinitialization similarly to regular sort. Fixes in the revised patch.

2) I think the purpose of sampleSlot should be explicitly documented
> (and I'm not sure "sample" is a good term here, as is suggest some sort
> of sampling (for example nodeAgg uses grp_firstTuple).
>

Yes, "sample" isn't a good term here. However, "first" isn't really
correct,
because we can skip some tuples from beginning of the group in
order to not form groups too frequently. I'd rather name it "pivot" tuple
slot.

> 3) skipCols/SkipKeyData seems a bit strange too, I think. I'd use
> PresortedKeyData or something like that.
>

Good point, renamed.

4) In cmpSortSkipCols, when checking if the columns changed, the patch
> does this:
>
> n = ((IncrementalSort *) node->ss.ps.plan)->skipCols;
>
> for (i = 0; i < n; i++)
> {
> ... check i-th key ...
> }
>
> My hunch is that checking the keys from the last one, i.e.
>
> for (i = (n-1); i >= 0; i--)
> {
> ....
> }
>
> would be faster. The reasoning is that with "ORDER BY a,b" the column
> "b" changes more often. But I've been unable to test this because of the
> segfault crashes.
>

Agreed.

5) The changes from
>
> if (pathkeys_contained_in(...))
>
> to
>
> n = pathkeys_common(pathkeys, subpath->pathkeys);
>
>
> if (n == 0)
>
> seem rather inconvenient to me, as it makes the code unnecessarily
> verbose. I wonder if there's a better way to deal with this.
>

I would rather say, that it changes from

if (pathkeys_contained_in(...))

to

n = pathkeys_common(pathkeys, subpath->pathkeys);

if (n == list_length(pathkeys))

I've introduced pathkeys_common_contained_in() which returns the same
result as pathkeys_contained_in(), but sets number of common pathkeys
to the last argument. It simplifies code a little bit. The name, probably,
could be improved.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company