Re: database/table snapshot

From: Asko Oja <ascoja(at)gmail(dot)com>
To: Grzegorz Jaśkiewicz <gryzman(at)gmail(dot)com>
Cc: General Postgres Mailing List <pgsql-general(at)postgresql(dot)org>
Subject: Re: database/table snapshot
Date: 2009-01-30 15:32:17
Message-ID: ecd779860901300732r31b2307cnb786ca54f3ea32da@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hello

We have script called cube_dispatcher in SkyTools that we we use for similar
sounding purpose. We must provide daily snapshots of changes to online
tables for Business Intelligence team. So we create trigger on every table
that needs to be handled that writes changed records into queue and then
cube_dispatcher writes these into daily inherited tables the BI team can
conveniently import. That also provides easy way for dropping processed
data.

PgQ: Cube Dispatcher
 Has url encoded events as data source and writes them into partitoned
tables in
target database. Logutriga is used to create events.
 Used to provide batches of data for business intelligence and data cubes.
 Only one instance of each record is stored. For example if record is
created and
then updated twice only latest version of record stays in that days table.
 Does not support deletes.

regards
Asko

On Fri, Jan 30, 2009 at 4:10 PM, Grzegorz Jaśkiewicz <gryzman(at)gmail(dot)com>wrote:

> Hey folks,
>
> I wonder, how would you guys approach table snapshots.
> So the thing is, I have table X (I have few more tables, but lets
> simplify that). That table is being replicated to X' on other server.
> Now I need an ability to see changes, say every interval T. So, the
> simplest solution would be to insert all that data to table Xs, with
> additional time column,
> but that obviously will take vast amounts of space. Is there any
> existing algorithm for doing that, or perhaps there's something in
> postgres that would allow me to do it - but I don't know about ?
> thx.
>
>
> --
> GJ
>
> --
> Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Harald Fuchs 2009-01-30 15:38:20 Re: Call volume query
Previous Message Hiroshi Inoue 2009-01-30 15:11:05 Re: encoding of PostgreSQL messages