Skip site navigation (1) Skip section navigation (2)

Re: Page-level version upgrade (was: Block-level CRC checks)

From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Greg Smith <greg(at)2ndquadrant(dot)com>
Cc: Bruce Momjian <bruce(at)momjian(dot)us>, Greg Stark <gsstark(at)mit(dot)edu>, decibel <decibel(at)decibel(dot)org>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Simon Riggs <simon(at)2ndquadrant(dot)com>, Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>, Aidan Van Dyk <aidan(at)highrise(dot)ca>, Alvaro Herrera <alvherre(at)commandprompt(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Page-level version upgrade (was: Block-level CRC checks)
Date: 2009-12-02 04:17:58
Message-ID: 603c8f070912012017q43fe4595pc9a0a9a82fd034b8@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Tue, Dec 1, 2009 at 10:45 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> Robert Haas wrote:
>>
>> If the pre-upgrade utility is something
>> that has to be run while the database is off-line, then it defeats the
>> point of an in-place upgrade.  If it can be run while the database is
>> up, I fear it will need to be deeply integrated into the server.  And
>> since we can't know the requirements for how much space to reserve
>> (and it needn't be a constant) until we design the new feature, this
>> will likely mean backpatching a rather large chunk of complex code,
>> which to put it mildly, is not the sort of thing we normally would
>> even consider.
>
> You're wandering into the sort of overdesign that isn't really needed yet.
>  For now, presume it's a constant amount of overhead, and that the release
> notes for the new version will say "configure the pre-upgrade utility and
> tell it you need <x> bytes of space reserved".  That's sufficient for the
> CRC case, right?  Needs a few more bytes per page, 8.5 release notes could
> say exactly how much.  Solve that before making things more complicated by
> presuming you need to solve the variable-size increase problem, too.  We'll
> be lucky to get the absolute simplest approach committed, you really need to
> have a big smoking gun to justify feature creep in this area.

Well, I think the best way to solve the problem is to design the
system in a way that makes it unnecessary to have a pre-upgrade tool
at all, by making the new PG version capable of handling page
expansion where needed.  I don't understand how putting that
functionality into the OLD PG version can be better.  But I may be
misunderstanding something.

> (If I had to shoot from the hip and design for the variable case, why not
> just make the thing that determines how much space a given page needs
> reserved a function the user can re-install with a smarter version?)

That's a pretty good idea.   I have no love of this pre-upgrade
concept, but if we're going to do it that way, then allowing someone
to load in a function to compute the required amount of free space to
reserve is a good thought.

>> I think a better approach is to support reading tuples
>> from old pages, but to write all new tuples into new pages.  A
>> full-table rewrite (like UPDATE foo SET x = x, CLUSTER, etc.) can be
>> used to propel everything to the new version, with the usual tricks
>> for people who need to rewrite the table a piece at a time.
>
> I think you're oversimplifying the operational difficulty of "the usual
> tricks".  This is a painful approach for the exact people who need this the
> most:  people with a live multi-TB installation they can't really afford to
> add too much load to.  The beauty of the in-place upgrade tool just
> converting pages as it scans through looking for them is that you can dial
> up its intensity to exactly how much overhead you can stand, and let it
> loose until it's done.

Fair enough.

...Robert

In response to

pgsql-hackers by date

Next:From: Robert HaasDate: 2009-12-02 04:19:00
Subject: Re: operator exclusion constraints
Previous:From: Robert HaasDate: 2009-12-02 04:04:34
Subject: Re: Page-level version upgrade (was: Block-level CRC checks)

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group