Re: 回复:Re: Cache relation sizes?

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
To: David Steele <david(at)pgmasters(dot)net>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, 陈佳昕(步真) <buzhen(dot)cjx(at)alibaba-inc(dot)com>
Subject: Re: 回复:Re: Cache relation sizes?
Date: 2021-03-11 08:34:54
Message-ID: CA+hUKGJSjTWDmCEppgKrE3GejDF-XZGF84Vjs+Z0UB9rCU858g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Mar 4, 2021 at 2:39 AM David Steele <david(at)pgmasters(dot)net> wrote:
> On 1/18/21 10:42 PM, 陈佳昕(步真) wrote:
> > I want to share a patch with you, I change the replacement algorithm
> > from fifo to a simple lru.
>
> What do you think of this change?

Ok, if I'm reading this right, it changes the replacement algorithm
from "clock" to "gclock". Gclock is like PostgreSQL's buffer pool,
except that here the usage count is limited with a GUC, instead of the
universal constant 5. Certainly gclock is better than clock. I used
the simplest algorithm I could think of, deliberately trying not to go
down the cache replacement algorithm rabbit hole. I think we should
match the buffer pool, however good/bad that may be, and adding a
usage count does seem to do that.

I had some trouble applying the patch, it seems to be corrupted, and
to apply on top of some other pach (looks like a struct moved from a
.c to a .h?), but I managed to figure it out and make the tests pass,
and I've attached a rebase.

> Also, your patch set from [1] no longer applies (and of course this
> latest patch is confusing the tester as well).

Rebased. The main change was commit bea449c6 "Optimize
DropRelFileNodesAllBuffers() for recovery.". With this rebase, that
optimisation now works in online, which is cool (by "online" I mean
not just in recovery).

Other changes in this version:

* Added spinlock backoff recommended by Andres in his review earlier.

Erm, that's it. There are still many review comments from Buzhen and
Andres to address. The main thing being: how can we clean these
objects in the background? That question being unanswered, there is a
0% chance of committing this in PostgreSQL 14. So I will now move
this to the next CF.

Thanks for the reviews so far!

Attachment Content-Type Size
v4-0001-WIP-Track-relation-sizes-in-shared-memory.patch text/x-patch 34.9 KB
v4-0002-WIP-Provide-a-lock-free-fast-path-for-smgrnblocks.patch text/x-patch 6.7 KB
v4-0003-update-fifo-to-lru-to-sweep-a-valid-cache.patch text/x-patch 5.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Langote 2021-03-11 08:42:43 Re: Allow batched insert during cross-partition updates
Previous Message Greg Nancarrow 2021-03-11 08:33:00 Re: Parallel INSERT (INTO ... SELECT ...)