Re: heap metapages

On 5/22/12 12:09 PM, Simon Riggs wrote:
> On 22 May 2012 13:52, Robert Haas<robertmhaas(at)gmail(dot)com> wrote:
>
>> It seems pretty clear to me that making pg_upgrade responsible for
>> emptying block zero is a non-starter. But I don't think that's a
>> reason to throw out the design; I think it's a problem we can work
>> around.
>
> I like your design better as well *if* you can explain how we can get
> to it. My proposal was a practical alternative that would allow the
> idea to proceed.

It occurred to me that having a metapage with information useful to recovery operations in *every segment* would be useful; it certainly seems worth the extra block. It then occurred to me that we've basically been stuck with 2 places to store relation data; either at the relation level in pg_class or on each page. Sometimes neither one is a good fit.

ISTM that a lot of problems we've faced in the past few years are because there's not a good abstraction between a (mostly) linear tuplespace and the physical storage that goes underneath it.

- pg_upgrade progress is blocked because we can't deal with a new page that's > BLKSZ
- There's no good way to deal with table (or worse, index) bloat
- There's no good way to add the concept of a heap metapage
- Forks are being used to store data that might not belong there only because there's no other choice (visibility info)

Would it make sense to take a step back and think about ways to abstract between logical tuplespace and physical storage? What if 1GB segments had their own metadata? Or groups of segments? Could certain operations that currently have to rewrite an entire table be changed so that they slowly moved pages from one group of segments to another, with a means of marking old pages as having been moved?

Einstein said that "problems cannot be solved by the same level of thinking that created them." Perhaps we're at the point where we need to take a step back from our current storage organization and look for a bigger picture?
--
Jim C. Nasby, Database Architect jim(at)nasby(dot)net
512.569.9461 (cell) http://jim.nasby.net