Re: Protect syscache from bloating with negative cache entries

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: andres(at)anarazel(dot)de
Cc: tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com, alvherre(at)2ndquadrant(dot)com, tomas(dot)vondra(at)2ndquadrant(dot)com, bruce(at)momjian(dot)us, robertmhaas(at)gmail(dot)com, tgl(at)sss(dot)pgh(dot)pa(dot)us, ideriha(dot)takeshi(at)jp(dot)fujitsu(dot)com, pgsql-hackers(at)lists(dot)postgresql(dot)org, michael(dot)paquier(at)gmail(dot)com, david(at)pgmasters(dot)net, craig(at)2ndquadrant(dot)com
Subject: Re: Protect syscache from bloating with negative cache entries
Date: 2019-02-20 04:14:35
Message-ID: 20190220.131435.10559958.horiguchi.kyotaro@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

At Thu, 14 Feb 2019 00:40:10 -0800, Andres Freund <andres(at)anarazel(dot)de> wrote in <20190214084010(dot)bdn6tmba2j7szo3m(at)alap3(dot)anarazel(dot)de>
> Hi,
>
> On 2019-02-13 15:31:14 +0900, Kyotaro HORIGUCHI wrote:
> > Instead, I added an accounting(?) interface function.
> >
> > | MemoryContextGettConsumption(MemoryContext cxt);
> >
> > The API returns the current consumption in this memory
> > context. This allows "real" memory accounting almost without
> > overhead.
>
> That's definitely *NOT* almost without overhead. This adds additional
> instructions to one postgres' hottest set of codepaths.

I'm not sure how much the two instructions in AllocSetAlloc
actually impacts, but I agree that it is doubtful that the
size-limit feature worth the possible slowdown in any extent.

# I faintly remember that I tried the same thing before..

> I think you're not working incrementally enough here. I strongly suggest
> solving the negative cache entry problem, and then incrementally go from
> there after that's committed. The likelihood of this patch ever getting
> merged otherwise seems extremely small.

Mmm. Scoping to the negcache prolem, my very first patch posted
two-years ago does that based on invalidation for pg_statistic
and pg_class, like I think Tom have suggested somewhere in this
thread.

https://www.postgresql.org/message-id/20161219.201505.11562604.horiguchi.kyotaro@lab.ntt.co.jp

This is completely different approach from the current shape and
it would be useless after pruning is introduced. So I'd like to
go for the generic pruning by age.

Difference from v15:

Removed AllocSet accounting stuff. We use approximate memory
size for catcache.

Removed prune-by-number(or size) stuff.

Adressing comments from Tsunakawa-san and Ideriha-san .

Separated catcache monitoring feature. (Removed from this set)
(But it is crucial to check this feature...)

Is this small enough ?

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
v16-0001-Add-dlist_move_tail.patch text/x-patch 1.2 KB
v16-0002-Remove-entries-that-haven-t-been-used-for-a-certain-.patch text/x-patch 24.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Haribabu Kommi 2019-02-20 04:16:48 Re: pg_basebackup ignores the existing data directory permissions
Previous Message Masahiko Sawada 2019-02-20 04:00:13 Re: Copy function for logical replication slots