Re: WIP: System Versioned Temporal Table

From: Corey Huinker <corey(dot)huinker(at)gmail(dot)com>
To: Simon Riggs <simon(dot)riggs(at)enterprisedb(dot)com>
Cc: Jaime Casanova <jcasanov(at)systemguards(dot)com(dot)ec>, vignesh C <vignesh21(at)gmail(dot)com>, Ibrar Ahmed <ibrar(dot)ahmad(at)gmail(dot)com>, Surafel Temesgen <surafel3000(at)gmail(dot)com>, Vik Fearing <vik(at)postgresfriends(dot)org>, Rémi Lapeyre <remi(dot)lapeyre(at)lenstra(dot)fr>, Ryan Lambert <ryan(at)rustprooflabs(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, Eli Marmor <eli(at)netmask(dot)it>, David Steele <david(at)pgmasters(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Georgios <gkokolatos(at)protonmail(dot)com>, Li Japin <japinli(at)hotmail(dot)com>
Subject: Re: WIP: System Versioned Temporal Table
Date: 2021-09-20 04:57:11
Message-ID: CADkLM=f_1oURdQYuBgxvJ0Gyk=38rtyn+4kPqicnLkB_p4gTAA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

>
> Thanks for giving this a lot of thought. When you asked the question
> the first time you hadn't discussed how that might work, but now we
> have something to discuss.
>

My ultimate goal is to unify this effort with the application period
effort. Step 1 in that was to understand what each was doing and why they
were doing it. If you check out the other thread, you'll see a highly
similar message that I sent over there.

> There are 3 implementation routes that I see, so let me explain so
> that others can join the discussion.
>
> 1. Putting all data in one table. This makes DROP SYSTEM VERSIONING
> effectively impossible. It requires access to the table to be
> rewritten to add in historical quals for non-historical access and it
> requires some push-ups around indexes. (The current patch adds the
> historic quals by kludging the parser, which is wrong place, since it
> doesn't work for joins etc.. However, given that issue, the rest seems
> to follow on naturally).
>
> 2. Putting data in a side table. This makes DROP SYSTEM VERSIONING
> fairly trivial, but it complicates many DDL commands (please make a
> list?) and requires the optimizer to know about this and cater to it,
> possibly complicating plans. Neither issue is insurmountable, but it
> becomes more intrusive.
>
> The current patch could go in either of the first 2 directions with
> further work.
>
> 3. Let the Table Access Method handle it. I call this out separately
> since it avoids making changes to the rest of Postgres, which might be
> a good thing, with the right TAM implementation.
>

I'd like to hear more about this idea number 3.

I could see value in allowing the history table to be a foreign table,
perhaps writing to csv/parquet/whatever files, and that sort of setup could
be persuasive to a regulator who wants extra-double-secret-proof that
auditing cannot be tampered with. But with that we'd have to give up the
relkind idea, which itself was going to be a cheap way to prevent updates
outside of the system triggers.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Corey Huinker 2021-09-20 05:09:46 Re: WIP: System Versioned Temporal Table
Previous Message Antonin Houska 2021-09-20 04:56:14 Re: POC: Cleaning up orphaned files using undo logs