pgsql: Change the way pre-reading in external sort's merge phase works.

From: Heikki Linnakangas <heikki(dot)linnakangas(at)iki(dot)fi>
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: Change the way pre-reading in external sort's merge phase works.
Date: 2016-10-03 10:38:53
Message-ID: E1br0e5-0003MY-Qs@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Change the way pre-reading in external sort's merge phase works.

Don't pre-read tuples into SortTuple slots during merge. Instead, use the
memory for larger read buffers in logtape.c. We're doing the same number
of READTUP() calls either way, but managing the pre-read SortTuple slots
is much more complicated. Also, the on-tape representation is more compact
than SortTuples, so we can fit more pre-read tuples into the same amount
of memory this way. And we have better cache-locality, when we use just a
small number of SortTuple slots.

Now that we only hold one tuple from each tape in the SortTuple slots, we
can greatly simplify the "batch memory" management. We now maintain a
small set of fixed-sized slots, to hold the tuples, and fall back to
palloc() for larger tuples. We use this method during all merge phases,
not just the final merge, and also when randomAccess is requested, and
also in the TSS_SORTEDONTAPE case. In other words, it's used whenever we
do an external sort.

Reviewed by Peter Geoghegan and Claudio Freire.

Discussion: <CAM3SWZTpaORV=yQGVCG8Q4axcZ3MvF-05xe39ZvORdU9JcD6hQ(at)mail(dot)gmail(dot)com>

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/e94568ecc10f2638e542ae34f2990b821bbf90ac

Modified Files
--------------
src/backend/utils/sort/logtape.c | 153 ++++-
src/backend/utils/sort/tuplesort.c | 1216 +++++++++++++-----------------------
src/include/utils/logtape.h | 2 +
3 files changed, 574 insertions(+), 797 deletions(-)

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Tom Lane 2016-10-03 12:51:49 Re: [COMMITTERS] pgsql: Copy-editing for contrib/pg_visibility documentation.
Previous Message Michael Paquier 2016-10-03 03:09:12 Re: [COMMITTERS] pgsql: Copy-editing for contrib/pg_visibility documentation.