Re: Protect syscache from bloating with negative cache entries

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>, Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>, David Steele <david(at)pgmasters(dot)net>, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>, Craig Ringer <craig(at)2ndquadrant(dot)com>, PostgreSQL mailing lists <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Protect syscache from bloating with negative cache entries
Date: 2017-12-01 21:40:23
Message-ID: 3544.1512164423@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Andres Freund <andres(at)anarazel(dot)de> writes:
> On 2017-12-01 16:20:44 -0500, Robert Haas wrote:
>> Well, yeah, that would be insane. But I think even something very
>> rough could work well enough. I think our goal should be to eliminate
>> cache entries that are have gone unused for many *minutes*, and
>> there's no urgency about getting it to any sort of exact value. For
>> non-idle backends, using the most recent statement start time as a
>> proxy would probably be plenty good enough. Idle backends might need
>> a bit more thought.

> Our timer framework is flexible enough that we can install a
> once-a-minute timer without much overhead. That timer could increment a
> 'cache generation' integer. Upon cache access we write the current
> generation into relcache / syscache (and potentially also plancache?)
> entries. Not entirely free, but cheap enough. In those once-a-minute
> passes entries that haven't been touched in X cycles get pruned.

I have no faith in either of these proposals, because they both assume
that the problem only arises over the course of many minutes. In the
recent complaint about pg_dump causing relcache bloat, it probably does
not take nearly that long for the bloat to occur.

Maybe you could make it work on the basis of number of cache accesses,
or some other normalized-to-workload-not-wall-clock time reference.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2017-12-01 21:45:34 Re: Protect syscache from bloating with negative cache entries
Previous Message Andres Freund 2017-12-01 21:31:13 Re: Protect syscache from bloating with negative cache entries