Re: Protect syscache from bloating with negative cache entries

At Tue, 5 Feb 2019 02:40:35 +0000, "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com> wrote in <0A3221C70F24FB45833433255569204D1FB93A16(at)G01JPEXMBYT05>
> From: bruce(at)momjian(dot)us [mailto:bruce(at)momjian(dot)us]
> > On Mon, Feb 4, 2019 at 08:23:39AM +0000, Tsunakawa, Takayuki wrote:
> > > Horiguchi-san, Bruce, all, So, why don't we make
> > > syscache_memory_target the upper limit on the total size of all
> > > catcaches, and rethink the past LRU management?
> >
> > I was going to say that our experience with LRU has been that the
> > overhead is not worth the value, but that was in shared resource cases,
> > which this is not.
>
> That's good news! Then, let's proceed with the approach involving LRU, Horiguchi-san, Ideriha-san.

If you mean accessed-time-ordered list of entries by "LRU", I
still object to involve it since it is too complex in searching
code paths. Invalidation would make things more complex. The
current patch sorts entries by ct->lastaccess and discards
entries not accessed for more than threshold, only at doubling
cache capacity. It is already a kind of LRU in behavior.

This patch intends not to let caches bloat by unnecessary
entries, which is negative ones at first, then less-accessed ones
currently. If you mean by "LRU" something to put a hard limit on
the number or size of a catcache or all caches, it would be
doable by adding sort phase before pruning, like
CatCacheCleanOldEntriesByNum() in the attached as a PoC (first
attched) as food for discussion.

With the second attached script, we can observe what is happening
from another session by the following query.

select relname, size, ntuples, ageclass from pg_stat_syscache where relname =' pg_statistic'::regclass;

> pg_statistic | 1041024 | 7109 | {{1,1109},{3,0},{30,0},{60,0},{90,6000},{0,0

On the other hand, differently from the original pruning, this
happens irrelevantly to hash resize so it will causes another
observable intermittent slowdown than rehashing.

The two should have the same extent of impact on performance when
disabled. I'll take numbers briefly using pgbench.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center