Re: Again, sorry, caching.

From: "Ross J(dot) Reedstrom" <reedstrm(at)rice(dot)edu>
To: Neil Conway <nconway(at)klamath(dot)dyndns(dot)org>
Cc: F Harvell <fharvell(at)fts(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Again, sorry, caching.
Date: 2002-03-20 16:34:00
Message-ID: 20020320163400.GC4567@rice.edu
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Mar 19, 2002 at 08:28:19PM -0500, Neil Conway wrote:
> On Tue, 2002-03-19 at 19:20, F Harvell wrote:
> > I feel that the caching should be SQL transparent. If it is
> > implemented reasonably well, the performance gain should be pretty
> > much universal.
>
> Well, the simple query cache scheme that is currently being proposed
> would use a byte-by-byte comparison of the incoming query. I think the
> consensus is that for a lot of workloads, this would be a bad idea.

Apparently, no one actually read _my_ proposal, they just replied to it.
All the arguements about if this kind of cache is any good have been
thrashed out, up-thread. Apparently Mr. Harvell didn't feel the need
to go back and read them. Going over them again is not productive -
the next step is to see if there is anything to actually _code_ here.

Caching is a hard problem, once you start digging into it. Going from
no cache to some cache is (almost) always a win, but multiple caches in
a datapath can interact in non-intuitive ways. And we _already_ have
several, well tuned, clever caches in place. Anything that messes with
them is going to be rejected, for sure.

What I proposed is a sort of compromise: it is clear to me that the core
developers are not very interested in the kind of cache Neil is talking
about above, and would rather see query caching done in the app. What I
proposed is extending the backends support for client-side caching, to
make it easier (or possible) for middleware to automate the task.

The bare bones are: flag a query in some way so the backend auto generates
the appropriate NOTIFY triggers, so the middleware can do proper cache
maintenance by LISTENing.

I think I'll go away and write up my compromise proposal a little more
clearly, and post it under a new subject, later. Perhaps we can then
have a productive discussion about _it_, and not rehash old arguments.

Ross

P.S.

HACKER sociological opinion below - feel free to skip -

There are only three reasons to discuss features on HACKERS: to
see if a proposed feature would be rejected, so you don't waste time
implementing it; to refine a proposed implementation, so it doesn't have
to be reworked; and to discuss an actual in-hand implementation. Notice
that there's no way to skip step one: if the CVS committers don't like
the feature, arguing for it on HACKERS won't make it magically better:
providing an implementation that doesn't do bad things _might_. And you
can always maintain an independent patch, or fork.

So, we have a number of people who think a query cache would be a
good idea. And core developers who are not convinced. I think one
of the reasons is that, while it might be useful in some situations
(even fairly common situations) it's neither elegant nor flexible. The
PostgreSQL project has a long tradition of turning down narrow, 'good
enough - it works for me' solutions, while looking for a better, more
inclusive solution. Sometimes this has been a problem with missing
features, but in the long run, it's been a win.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Hannu Krosing 2002-03-20 16:40:16 Re: XML
Previous Message Thomas Lockhart 2002-03-20 15:24:17 Re: Notify argument?