Re: Page-level version upgrade (was: Block-level CRC checks)

Robert Haas wrote:
> If the pre-upgrade utility is something
> that has to be run while the database is off-line, then it defeats the
> point of an in-place upgrade. If it can be run while the database is
> up, I fear it will need to be deeply integrated into the server. And
> since we can't know the requirements for how much space to reserve
> (and it needn't be a constant) until we design the new feature, this
> will likely mean backpatching a rather large chunk of complex code,
> which to put it mildly, is not the sort of thing we normally would
> even consider.
You're wandering into the sort of overdesign that isn't really needed
yet. For now, presume it's a constant amount of overhead, and that the
release notes for the new version will say "configure the pre-upgrade
utility and tell it you need <x> bytes of space reserved". That's
sufficient for the CRC case, right? Needs a few more bytes per page,
8.5 release notes could say exactly how much. Solve that before making
things more complicated by presuming you need to solve the variable-size
increase problem, too. We'll be lucky to get the absolute simplest
approach committed, you really need to have a big smoking gun to justify
feature creep in this area.

(If I had to shoot from the hip and design for the variable case, why
not just make the thing that determines how much space a given page
needs reserved a function the user can re-install with a smarter version?)
> I think a better approach is to support reading tuples
> from old pages, but to write all new tuples into new pages. A
> full-table rewrite (like UPDATE foo SET x = x, CLUSTER, etc.) can be
> used to propel everything to the new version, with the usual tricks
> for people who need to rewrite the table a piece at a time.
I think you're oversimplifying the operational difficulty of "the usual
tricks". This is a painful approach for the exact people who need this
the most: people with a live multi-TB installation they can't really
afford to add too much load to. The beauty of the in-place upgrade tool
just converting pages as it scans through looking for them is that you
can dial up its intensity to exactly how much overhead you can stand,
and let it loose until it's done.

--
Greg Smith 2ndQuadrant Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com www.2ndQuadrant.com