Re: Protect syscache from bloating with negative cache entries

Thank you for the discussion.

At Tue, 20 Dec 2016 15:10:21 -0500, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote in <23492(dot)1482264621(at)sss(dot)pgh(dot)pa(dot)us>
> The bigger picture here though is that we used to have limits on syscache
> size, and we got rid of them (commit 8b9bc234a, see also
> https://www.postgresql.org/message-id/flat/5141.1150327541%40sss.pgh.pa.us)
> not only because of the problem you mentioned about performance falling
> off a cliff once the working-set size exceeded the arbitrary limit, but
> also because enforcing the limit added significant overhead --- and did so
> whether or not you got any benefit from it, ie even if the limit is never
> reached. Maybe the present patch avoids imposing a pile of overhead in
> situations where no pruning is needed, but it doesn't really look very
> promising from that angle in a quick once-over.

Indeed. As mentioned in the mail at the beginning of this thread,
it hits the whole-cache scanning if at least one negative cache
exists even it is not in a relation with the target relid, and it
can be significantly long on a fat cache.

Lists of negative entries like CatCacheList would help but needs
additional memeory.

> BTW, I don't see the point of the second patch at all? Surely, if
> an object is deleted or updated, we already have code that flushes
> related catcache entries. Otherwise the caches would deliver wrong
> data.

Maybe you take the patch wrongly. Negative entires won't be
flushed by any means. Deletion of a namespace causes cascaded
object deletion according to dependency then finaly goes to
non-neative cache invalidation. But a removal of *negative
entries* in RELNAMENSP won't happen.

The test script for the case (gen2.pl) does the following thing,

CREATE SCHEMA foo;
SELECT * FROM foo.invalid;
DROP SCHEMA foo;

Removing the schema foo leaves a negative cache entry for
'foo.invalid' in RELNAMENSP.

However, I'm not sure the above situation happens so frequent
that it is worthwhile to amend.

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center