Re: [WIP] In-place upgrade

Heikki Linnakangas napsal(a):
> Tom Lane wrote:
>> I think we can have a notion of pre-upgrade maintenance, but it would
>> have to be integrated into normal operations. For instance, if
>> conversion to 8.4 requires extra free space, we'd make late releases
>> of 8.3.x not only be able to force that to occur, but also tweak the
>> normal code paths to maintain that minimum free space.
>
> Agreed, the backend needs to be modified to reserve the space.
>
>> The full concept as I understood it (dunno why Bruce left all these
>> details out of his message) went like this:
>>
>> * Add a "format serial number" column to pg_class, and probably also
>> pg_database. Rather like the frozenxid columns, this would have the
>> semantics that all pages in a relation or database are known to have at
>> least the specified format number.
>>
>> * There would actually be two serial numbers per release, at least for
>> releases where pre-update prep work is involved --- for instance,
>> between 8.3 and 8.4 there'd be an "8.3-and-a-half" format which is
>> 8.3 but known ready to update to 8.4 (eg, enough free space available).
>> Minor releases of 8.3 that appear with or subsequent to 8.4 release
>> understand the "half" format number and how to upgrade to it.
>>
>> * VACUUM would be empowered, in the same way as it handles frozenxid
>> maintenance, to update any less-than-the-latest-version pages and then
>> fix the pg_class and pg_database entries.
>>
>> * We could mechanically enforce that you not update until the database
>> is ready for it by checking pg_database.datformatversion during
>> postmaster startup.
>
> Adding catalog columns seems rather complicated, and not back-patchable.
> Not backpatchable means that we'd need to be sure now that the format
> serial numbers are enough for the upcoming 8.4-8.5 upgrade.

Reloptions is suitable for keeping amount of reserver space. And it can be back
ported into 8.3 and 8.2. And of course there is no problem to convert 8.1->8.2.

For backported branch would be better to combine internal modification -
preserve space and e.g. store procedure which check all relations.

In the 8.4 and newer pg_class could be extended for new attributes.

> I imagined that you would have just a single cluster-wide variable, a
> GUC perhaps, indicating how much space should be reserved by
> updates/inserts.

You sometimes need different reserved size for different type of relation. For
example on 32bit x86 you don't need reserve space for heap but you need do it
for indexes (between v3->v4). Better is to use reloptions and pre-upgrade
procedure sets this information correctly.

> Then you'd have an additional program, perhaps a new
> contrib module, that sets the variable to the right value for the
> version you're upgrading, and scans through all tables, moving tuples so
> that every page has enough free space for the upgrade. After that's
> done, it'd set a flag in the data directory indicating that the cluster
> is ready for upgrade.

I prefer to have this information in pg_class. It is accessible by SQL commands.
pg_class should also contains information about last checked page to prevent
repeatable check on very large tables.

> The tool could run concurrently with normal activity, so you could just
> let it run for as long as it takes.

Agree.

Zdenek

--
Zdenek Kotala Sun Microsystems
Prague, Czech Republic http://sun.com/postgresql