Re: Global shared meta cache

From: Andres Freund <andres(at)anarazel(dot)de>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru>, "Ideriha, Takeshi" <ideriha(dot)takeshi(at)jp(dot)fujitsu(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Global shared meta cache
Date: 2018-07-05 18:04:28
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


On 2018-07-05 10:00:13 -0400, Robert Haas wrote:
> I think we need to take a little bit broader view of this problem.
> For instance, maybe we could have backend-local caches that are kept
> relatively small, and then a larger shared cache that can hold more
> entries.

I think it's pretty much *required* that we have that. Not just for
speed, but for correctness. It'll otherwise be very hard to deal with
transactional DDL. I'm pretty sure that we'll have to have everything
modified by the local transaction in that cache. There's a lot of
interesting additional problems with snapshots stil, but that seems like
a baseline requirement.

> Obviously, loading an entry into our backend-private cache must be a
> LOT slower than consulting one.

I'm not actually sure it makes *that* much of a difference, if the local
cache is small.

> I would guess that we'd want to try to use something like the
> st_changecount protocol to make reads very cheap and writes
> comparatively more expensive, since by and large cache invalidations
> aren't that frequent.

That strikes me as hard to get right and fatal to get wrong. I suspect
normal RW locking ought to do fine.

> One experiment I think would be interesting is to study how much
> catcache traffic we're actually generating and see if we can find any
> way to reduce it. For instance, somebody could write code to record
> the file and line number for every catcache lookup and then run a test
> workload (or several test workloads). That might give us some ideas
> about stuff we could just decide not to cache, especially if we also
> knew how large each cache ended up being.

We definitely do a lot of redundant lookups for the same entries.


Andres Freund

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2018-07-05 18:22:32 Re: pgsql: Fix "base" snapshot handling in logical decoding
Previous Message Andrey Borodin 2018-07-05 17:27:29 Re: Covering GiST indexes