Re: TOAST versus VACUUM, or "missing chunk number 0 for toast value" identified

On Wed, Oct 26, 2011 at 4:24 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Even given your recent changes to reduce the overhead of checking for
> sinval messages, I'm not sure that it'd be practical to move the sinval
> message processing to just-before-we-look-up-a-cache-entry.  Right now,
> we do AcceptInvalidationMessages basically once per table per query
> (or maybe it's twice or so, but anyway a very small multiplier on that).
> If we try to do it every time through SearchSysCache, we are probably
> talking two to three orders of magnitude more checks, which ISTM is
> certain to push the sinval queue back up to the top of the heap for
> contention.

Initially I provided an implementation of eager locking, as Robert
suggests, but the above is a great argument against doing it that way.

Incidentally, I'd like to focus on the causal reason for these
problems. We have static type definitions, which allow us to
completely avoid dynamic type management at runtime. That is a
performance advantage for us in normal running, but it can also give
operational problems when we look to make changes.

> But in any case, this isn't the core of the problem.  The real point
> here is that we need a guarantee that a syscache entry we're going to
> use is/was valid as of some suitable time point later than the start of
> our current transaction.  (Once we have taken a snapshot, VACUUM will
> know that it can't remove any tuples that were deleted after the time of
> that snapshot; so even for SnapshotNow fetches, it's important to have
> an MVCC snapshot to protect toast-table dereferences.)  Perhaps rather
> than tying the problem into SearchSysCache, we should attach the
> overhead to GetTransactionSnapshot, which is called appealingly few
> times per query.  But right offhand it seems like that only protects us
> against the toast-tuple-deletion problem, not against the more general
> one of getting a stale view of the status of some relation.

Do we need to guarantee that? It seems strange to me to use the words
cache and strict in the same sentence.

I'm sure there are many points in the code where strictness is
required though also in many cases, it should be acceptable to say
that a cache miss is not a problem, so does not require strict
guarantees.

Do we need a redesign? Or do we need a way to use
relaxation/optimistic techniques.

I'm aware that it could be a huge undertaking to examine all call
points. If you have any ideas of where to investigate or parts of the
problem to assist with, I'll be happy to work on this. This seems
important to me.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services