RE: Protect syscache from bloating with negative cache entries

From: "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>
To: 'Kyotaro HORIGUCHI' <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
Cc: "alvherre(at)2ndquadrant(dot)com" <alvherre(at)2ndquadrant(dot)com>, "tomas(dot)vondra(at)2ndquadrant(dot)com" <tomas(dot)vondra(at)2ndquadrant(dot)com>, "bruce(at)momjian(dot)us" <bruce(at)momjian(dot)us>, "andres(at)anarazel(dot)de" <andres(at)anarazel(dot)de>, "robertmhaas(at)gmail(dot)com" <robertmhaas(at)gmail(dot)com>, "tgl(at)sss(dot)pgh(dot)pa(dot)us" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "Ideriha, Takeshi" <ideriha(dot)takeshi(at)jp(dot)fujitsu(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, "michael(dot)paquier(at)gmail(dot)com" <michael(dot)paquier(at)gmail(dot)com>, "david(at)pgmasters(dot)net" <david(at)pgmasters(dot)net>, "craig(at)2ndquadrant(dot)com" <craig(at)2ndquadrant(dot)com>
Subject: RE: Protect syscache from bloating with negative cache entries
Date: 2019-02-14 07:23:18
Message-ID: 0A3221C70F24FB45833433255569204D1FB99888@G01JPEXMBYT05
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

From: Kyotaro HORIGUCHI [mailto:horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp]
> It is too complex as I was afraid. The indirect calls causes siginicant
> degradation. (Anyway the previous code was bogus in that it passes
> CACHELINEALIGN'ed pointer to get_chunk_size..)
>
> Instead, I added an accounting(?) interface function.
>
> | MemoryContextGettConsumption(MemoryContext cxt);
>
> The API returns the current consumption in this memory context. This allows
> "real" memory accounting almost without overhead.

That looks like a great idea! Actually, I was thinking of using MemoryContextStats() or its new lightweight variant to get the used amount, but I was afraid it would be too costly to call in catcache code. You are smarter, and I was just stupid.

> (2) Another new patch v15-0005 on top of previous design of
> limit-by-number-of-a-cache feature converts it to
> limit-by-size-on-all-caches feature, which I think is
> Tsunakawa-san wanted.

Thank you very, very much! I look forward to reviewing v15. I'll be away from the office tomorrow, so I'd like to review it on this weekend or the beginning of next week. I've confirmed and am sure that 0001 can be committed.

> As far as I can see no significant degradation is found in usual (as long
> as pruning doesn't happen) code paths.
>
> About the new global-size based evicition(2), cache entry creation becomes
> slow after the total size reached to the limit since every one new entry
> evicts one or more old (=
> not-recently-used) entries. Because of not needing knbos for each cache,
> it become far realistic. So I added documentation of
> "catalog_cache_max_size" in 0005.

Could you show us the comparison of before and after the pruning starts, if you already have it? If you lost the data, I'm OK to see the data after the code review.

Regards
Takayuki Tsunakawa

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Noah Misch 2019-02-14 07:26:23 Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding
Previous Message Peter Geoghegan 2019-02-14 07:21:39 Re: WAL insert delay settings