Re: Protect syscache from bloating with negative cache entries

At Fri, 1 Dec 2017 14:12:20 -0800, Andres Freund <andres(at)anarazel(dot)de> wrote in <20171201221220(dot)z5e6wtlpl264wzik(at)alap3(dot)anarazel(dot)de>
> On 2017-12-01 17:03:28 -0500, Tom Lane wrote:
> > Andres Freund <andres(at)anarazel(dot)de> writes:
> > > On 2017-12-01 16:40:23 -0500, Tom Lane wrote:
> > >> I have no faith in either of these proposals, because they both assume
> > >> that the problem only arises over the course of many minutes. In the
> > >> recent complaint about pg_dump causing relcache bloat, it probably does
> > >> not take nearly that long for the bloat to occur.
> >
> > > To me that's a bit of a different problem than what I was discussing
> > > here. It also actually doesn't seem that hard - if your caches are
> > > growing fast, you'll continually get hash-resizing of the
> > > various. Adding cache-pruning to the resizing code doesn't seem hard,
> > > and wouldn't add meaningful overhead.
> >
> > That's an interesting way to think about it, as well, though I'm not
> > sure it's quite that simple. If you tie this to cache resizing then
> > the cache will have to grow up to the newly increased size before
> > you'll prune it again. That doesn't sound like it will lead to nice
> > steady-state behavior.
>
> Yea, it's not perfect - but if we do pruning both at resize *and* on
> regular intervals, like once-a-minute as I was suggesting, I don't think
> it's that bad. The steady state won't be reached within seconds, true,
> but the negative consequences of only attempting to shrink the cache
> upon resizing when the cache size is growing fast anyway doesn't seem
> that large.
>
> I don't think we need to be super accurate here, there just needs to be
> *some* backpressure.
>
> I've had cases in the past where just occasionally blasting the cache
> away would've been good enough.

Thank you very much for the valuable suggestions. I still would
like to solve this problem and the
a-counter-freely-running-in-minute(or several seconds)-resolution
and pruning-too-long-unaccessed-entries-on-resizing seems to me
to work enough for at least several known bloat cases. This still
has a defect that this is not workable for a very quick
bloating. I'll try thinking about the remaining issue.

If no one has immediate objection to the direction, I'll come up
with an implementation.

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center