Re: Dynamic Partitioning using Segment Visibility Maps

From: Simon Riggs <simon(at)2ndquadrant(dot)com>
To: Richard Huxton <dev(at)archonet(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Dynamic Partitioning using Segment Visibility Maps
Date: 2008-01-04 10:40:17
Message-ID: 1199443217.18598.32.camel@ebony.site
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, 2008-01-04 at 10:22 +0000, Richard Huxton wrote:
> Simon Riggs wrote:
> > We would keep a dynamic visibility map at *segment* level, showing which
> > segments have all rows as 100% visible. No freespace map data would be
> > held at this level.
>
> Small dumb-user question.
>
> I take it you've considered some more flexible consecutive-run-of-blocks
> unit of flagging rather than file-segments. That obviously complicates
> the tracking but means you can cope with infrequent updates as well as
> mark most of the "most recent" segment for log-style tables.

I'm writing the code to abstract that away, so yes.

Now you mention it, it does seem straightforward to have a table storage
parameter for partition size, which defaults to 1GB. The partition size
is simply a number of consecutive blocks, as you say.

The smaller the partition size the greater the overhead of managing it.
Also I've been looking at read-only tables and compression, as you may
know. My idea was that in the future we could mark segments as either
- read-only
- compressed
- able to be shipped off to hierarchical storage

Those ideas work best if the partitioning is based around the physical
file sizes we use for segments.

Changing the partition size would simply reset the visibility map for
that table, in its easiest implementation.

--
Simon Riggs
2ndQuadrant http://www.2ndQuadrant.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Glyn Astill 2008-01-04 10:50:20 Problem with PgTcl auditing function on trigger
Previous Message Richard Huxton 2008-01-04 10:22:40 Re: Dynamic Partitioning using Segment Visibility Maps