Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Ryan Lambert <ryan(at)rustprooflabs(dot)com>
Cc: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Stephen Frost <sfrost(at)snowman(dot)net>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Joe Conway <mail(at)joeconway(dot)com>, Antonin Houska <ah(at)cybertec(dot)at>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>, "Moon, Insung" <Moon_Insung_i3(at)lab(dot)ntt(dot)co(dot)jp>, Ibrar Ahmed <ibrar(dot)ahmad(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)
Date: 2019-07-11 15:10:14
Message-ID: 20190711151014.yjwlblwywsc7r5up@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Jul 11, 2019 at 06:45:36AM -0600, Ryan Lambert wrote:
> > >>Uh, what if a transaction modifies page 0 and page 1 of the same table
>
> > >>--- don't those pages have the same LSN.
> > >
> > >No, because WAL being a physical change log, each page gets its own
> > >WAL record with its own LSN.
> > >
> >
> > What if you have wal_log_hints=off? AFAIK that won't change the page LSN.
>
> >
>
> > Alvaro suggested elsewhere that we require checksums for these, which
> > would also force wal_log_hints to be on, and therefore the LSN would
> > change.
>  
> Yes, it sounds like the agreement was LSN is unique when wal_log_hints is on. 
> I don't know enough about the internals to know if pg_class.oid is also needed
> or not.

Well, so, as far as we know now, every change to a heap/index page
advances the LSN, except for hint bits, which we can force to advance
LSN via wal_log_hints. We automatically enable wal_log_hints for
checksums, so we can easily enable it automatically for encrypted
clusters.

I assume the LSN used for 8k pages and the segment numbers (for WAL) do
not overlap in numbering, for our nonce.

I think we will eventually have to have this setup verified by security
experts but I think we need to work through what is possible using
existing Postgres facilities before we present a possible solution.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Alexander Korotkov 2019-07-11 15:23:20 Re: SQL/JSON path issues/questions
Previous Message Alexander Korotkov 2019-07-11 15:08:58 Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.