More ideas for speeding up sorting

From: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: More ideas for speeding up sorting
Date: 2016-09-09 13:14:34
Message-ID: d89aea6d-2b18-f61d-455e-e6790db636d3@iki.fi
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

A few things caught my eye while hacking on the latest round of sorting
patches. Starting a new thread because these are orthogonal to the
things discussed on the other threads:

1. SortTuple.tupindex is not used when the sort fits in memory. If we
also got rid of the isnull1 flag, we could shrink SortTuple from 24
bytes to 16 bytes (on 64-bit systems). That would allow you to pack more
tuples into memory, which generally speeds things up. We could for
example steal the least-significant bit from the 'tuple' pointer for
that, because the pointers are always at least 4-byte aligned. (But see
next point)

2. We could easily categorize incoming tuples as the come in, into those
that have a leading NULL, and others. If we kept them in separate
arrays, or perhaps grow memtuples from bottom-up for non-NULLs and from
top-down for NULLs, we wouldn't need to store, or compare, the 'isnull1'
flag at all, in the quicksort.

3. In the single-Datum case, i.e. tuplesort_begin_datum(), we could just
keep a counter of how many NULLs we saw, and not store them at all.
That's quite a special case, but it's simple enough that it might be
worth it.

I'm not going to pursue these right now, but might be good projects for
someone.

- Heikki

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Steele 2016-09-09 13:18:29 Re: PATCH: Exclude additional directories in pg_basebackup
Previous Message Heikki Linnakangas 2016-09-09 12:48:46 Re: Tuplesort merge pre-reading