Live sort-of-'warehousing' database how-to?

From: Mario Splivalo <mario(dot)splivalo(at)megafon(dot)hr>
To: pgsql-admin(at)postgresql(dot)org
Subject: Live sort-of-'warehousing' database how-to?
Date: 2010-03-31 14:19:46
Message-ID: 4BB35A02.5060100@megafon.hr
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Suppose I have 'stupid' database with just one tables, like this:

CREATE TABLE messages (
message_id uuid NOT NULL PRIMARY KEY,
message_time_created timestamp with time zone NOT NULL,
message_phone_number character varying NOT NULL,
message_state type_some_state_enum NOT NULL,
message_value numeric(10,4)
)

Now, let's say that I end up with around 1.000.000 records each week. I
actually need just last week or two worth of data for the whole system
to function normaly.

But, sometimes I do need to peek into 'messages' for some old message,
let's say a year old.

So I would like to keep 'running' messages on the 'main' server, and
keep there a month worth of data. On the 'auxiliary' server I'd like to
keep all the data. (Messages on the 'auxiliary' server are in the final
state, no change to that data will ever be made).

Is there a solution to achieve something like that. It is fairly easy to
implement something like

INSERT INTO auxilary.database.messages
SELECT * FROM main.database.messagaes
WHERE message_id NOT IN (SELECT message_id FROM
auxilary.database.messages....)

using python/dblink or something like that. But, is there already a
solution that would do something like that?

Or is there a better way to achieve desired functionality?

Mike

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Tom Lane 2010-03-31 14:28:31 Re: Statistics Collector not collecting server activities
Previous Message Gnanakumar 2010-03-31 13:22:53 Re: Statistics Collector not collecting server activities