RE: Global shared meta cache

From: "Ideriha, Takeshi" <ideriha(dot)takeshi(at)jp(dot)fujitsu(dot)com>
To: 'Amit Langote' <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>, Robert Haas<robertmhaas(at)gmail(dot)com>, Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru>,"andres(at)anarazel(dot)de" <andres(at)anarazel(dot)de>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: Global shared meta cache
Date: 2018-10-03 06:17:35
Message-ID: 4E72940DA2BF16479384A86D54D0988A6F1BC271@G01JPEXMBKW04
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

Hi,

Thank you for the previous discussion while ago.
I’m afraid I haven't replied to all.

To move forward this development I attached a PoC patch.

I introduced a guc called shared_catacache_mem to specify
how much memory is supposed be allocated on the shared memory area.
It defaults to zero, which indicates that no catalog cache is shared
but allocated on each backend MemoryContext (same as current Postgres).

At this moment this patch only allocates catalog cache header and CatCache data on the shared memory area.
It doesn't do much work, just starting and stopping postgres server with shared_catcache_mem non-zero.

Shared version CatCacheHdr is put on the postgres-initialized shared memory so that backends attach it
and build SysCache[] to store pointers of CatCache.
Each CatCache, CatCTup and CacCList is also allocated on the shared memory area,
where the limit size is the value of shared_catcache_mem and backed by DSA.
This area is first created at the postgres-initialized shared memory and re-initialized as DSA area
because the address of postgres-initialized shared area does not change among different process
and hopefully makes it easy to handle pointers on the shared memory.
(Though I'm still struggling to grasp the idea of DSA and underlying DSM..)

The followings are major items I haven't touched:
- make hash table of each CatCache shared, which I'm going to take advantage of dshash
- how to evict shared cache (LRU mechanism)
- how to treat cache visibility and invalidation coming from transactions including DDL
- how to alleviate the slowness compared to current PostgreSQL
- make relcache shared as well as catcache

If you have any insights/reactions/suggestions, please feel free to comment.

====================
Takeshi Ideriha
Fujitsu Limited

Attachment Content-Type Size
0001-PoC-Allocate-catcache-on-the-shared-memory.patch application/octet-stream 17.8 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2018-10-03 06:40:03 Re: [WIP PATCH] Index scan offset optimisation using visibility map
Previous Message Andres Freund 2018-10-03 06:09:36 Re: Performance improvements for src/port/snprintf.c