Re: Protect syscache from bloating with negative cache entries

At Tue, 12 Feb 2019 20:36:28 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote in <20190212(dot)203628(dot)118792892(dot)horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
> > (4)
> > + hash_size = cp->cc_nbuckets * sizeof(dlist_head);
> > + tupsize = sizeof(CatCTup) + MAXIMUM_ALIGNOF + dtp->t_len;
> > + tupsize = sizeof(CatCTup);
> >
> > GetMemoryChunkSpace() should be used to include the memory context overhead. That's what the files in src/backend/utils/sort/ do.
>
> Thanks. Done. Include bucket and cache header part but still
> excluding clist. Renamed from tupsize to memusage.

It is too complex as I was afraid. The indirect calls causes
siginicant degradation. (Anyway the previous code was bogus in
that it passes CACHELINEALIGN'ed pointer to get_chunk_size..)

Instead, I added an accounting(?) interface function.

| MemoryContextGettConsumption(MemoryContext cxt);

The API returns the current consumption in this memory
context. This allows "real" memory accounting almost without
overhead.

(1) New patch v15-0002 adds accounting feature to MemoryContext.
(It adds this feature only to AllocSet, if this is acceptable
it can be extended to other allocators.)

(2) Another new patch v15-0005 on top of previous design of
limit-by-number-of-a-cache feature converts it to
limit-by-size-on-all-caches feature, which I think is
Tsunakawa-san wanted.

As far as I can see no significant degradation is found in usual
(as long as pruning doesn't happen) code paths.

About the new global-size based evicition(2), cache entry
creation becomes slow after the total size reached to the limit
since every one new entry evicts one or more old (=
not-recently-used) entries. Because of not needing knbos for each
cache, it become far realistic. So I added documentation of
"catalog_cache_max_size" in 0005.

About the age-based eviction, the bulk eviction seems to take a a
bit long time but it happnes instead of hash resizing so the user
doesn't observe additional slowdown. On the contrary the pruning
can avoid rehashing scanning the whole cache. I think it is the
gain seen in the Tomas' experiment.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center