Re: Page-level version upgrade (was: Block-level CRC checks)

On Wed, Dec 2, 2009 at 1:08 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> Robert Haas wrote:
>>
>> The problem I'm referring to is that there is no guarantee that you
>> would be able predict how much space to reserve.  In a case like CRCs,
>> it may be as simple as "4 bytes".  But what if, say, we switch to a
>> different compression algorithm for inline toast?
>
> Upthread, you made a perfectly sensible suggestion:  use the CRC addition as
> a test case to confirm you can build something useful that allowed slightly
> more complicated in-place upgrades than are supported now.  This requires
> some new code to do tuple shuffling, communicate reserved space, etc.  All
> things that seem quite sensible to have available, useful steps toward a
> more comprehensive solution, and an achievable goal you wouldn't even have
> to argue about.
>
> Now, you're wandering us back down the path where we have to solve a
> "migrate TOAST changes" level problem in order to make progress.  Starting
> with presuming you have to solve the hardest possible issue around is the
> documented path to failure here.  We've seen multiple such solutions before,
> and they all had trade-offs deemed unacceptable:  either a performance loss
> for everyone (not just people upgrading), or unbearable code complexity.
>  There's every reason to believe your reinvention of the same techniques
> will suffer the same fate.

Just to set the record straight, I don't intend to work on this
problem at all (unless paid, of course). And I'm perfectly happy to
go with whatever workable solution someone else comes up with. I'm
just offering opinions on what I see as the advantages and
disadvantages of different approaches, and anyone is working on this
is more than free to ignore them.

> Some additional catalog support was suggested to mark what the pre-upgrade
> utility had processed.   I'm sure I could find the messages about again if I
> had to.

And that's a perfectly sensible solution, except that adding a catalog
column to 8.4 at this point would force initdb, so that's a
non-starter. I suppose we could shoehorn it into the reloptions.

> Also, your logic seems to presume that no backports are possible to the old
> server.

The problem on the table at the moment is that the proposed CRC
feature will expand every page by a uniform amount - so in this case a
fixed-space-per-page reservation utility would be completely adequate.
Does anyone think this is a realistic thing to backport to 8.4?

...Robert