Re: memory leak in e94568ecc10 (pre-reading in external sort)

From: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
To: Peter Geoghegan <pg(at)heroku(dot)com>
Cc: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: memory leak in e94568ecc10 (pre-reading in external sort)
Date: 2016-10-10 12:21:24
Message-ID: 0f607c4b-df23-353e-bf56-c0389d28495f@iki.fi
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 10/06/2016 06:44 PM, Peter Geoghegan wrote:
> While the fix you pushed was probably a good idea anyway, I still
> think you should not use swhichtate->maxTapes to exhaustively call
> LogicalTapeAssignReadBufferSize() on every tape, even non-active
> tapes. That's the confusing part.

Admittedly that's confusing. Thinking about this some more, I came up
with the attached. I removed the separate
LogicalTapeAssignReadBufferSize() call altogether - the read buffer size
is now passed as argument to the LogicalTapeRewindForRead() call.

I split LogicalTapeRewind() into two: LogicalTapeRewindForRead() and
LogicalTapeRewindForWrite(). A boolean argument like 'forWrite' is
mentally challenging, because when reading a call site, you have to
remember which way round the boolean is. And now you also pass the extra
buffer_size argument when rewinding for reading, but not when writing.

I gave up on the logic to calculate the quotient and reminder of the
available memory, and assigning one extra buffer to some of the tapes.
I'm now just rounding it down to the nearest BLCKSZ boundary. That
simplifies the code further, although we're now not using all the memory
we could. I'm pretty sure that's OK, although I haven't done any
performance testing of this.

- Heikki

Attachment Content-Type Size
0001-Simplify-the-code-for-logical-tape-read-buffers.patch text/x-patch 17.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2016-10-10 12:37:37 Using pg_ctl promote -w in TAP tests
Previous Message Craig Ringer 2016-10-10 12:06:58 Re: Is it time to kill support for very old servers?