Skip site navigation (1) Skip section navigation (2)

Re: Dynamic Partitioning using Segment Visibility Maps

From: Simon Riggs <simon(at)2ndquadrant(dot)com>
To: Richard Huxton <dev(at)archonet(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Dynamic Partitioning using Segment Visibility Maps
Date: 2008-01-04 10:40:17
Message-ID: 1199443217.18598.32.camel@ebony.site (view raw or flat)
Thread:
Lists: pgsql-hackers
On Fri, 2008-01-04 at 10:22 +0000, Richard Huxton wrote:
> Simon Riggs wrote:
> > We would keep a dynamic visibility map at *segment* level, showing which
> > segments have all rows as 100% visible. No freespace map data would be
> > held at this level. 
> 
> Small dumb-user question.
> 
> I take it you've considered some more flexible consecutive-run-of-blocks 
> unit of flagging rather than file-segments. That obviously complicates 
> the tracking but means you can cope with infrequent updates as well as 
> mark most of the "most recent" segment for log-style tables.

I'm writing the code to abstract that away, so yes.

Now you mention it, it does seem straightforward to have a table storage
parameter for partition size, which defaults to 1GB. The partition size
is simply a number of consecutive blocks, as you say.

The smaller the partition size the greater the overhead of managing it.
Also I've been looking at read-only tables and compression, as you may
know. My idea was that in the future we could mark segments as either
- read-only 
- compressed
- able to be shipped off to hierarchical storage

Those ideas work best if the partitioning is based around the physical
file sizes we use for segments.

Changing the partition size would simply reset the visibility map for
that table, in its easiest implementation.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


In response to

Responses

pgsql-hackers by date

Next:From: Glyn AstillDate: 2008-01-04 10:50:20
Subject: Problem with PgTcl auditing function on trigger
Previous:From: Richard HuxtonDate: 2008-01-04 10:22:40
Subject: Re: Dynamic Partitioning using Segment Visibility Maps

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group