Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019 at 9:44 AM Bruce Momjian <bruce(at)momjian(dot)us> wrote:

> > Checking that all buffers using a single LSN are from the same
> > relation would be a good idea but I think it's hard to test it and
> > regard the test result as okay. Even if we passed 'make checkworld',
> > it might still be possible to happen. And even assertion failures
>
> Yes, the problem is that if you embed the relfilenode or tablespace or
> database in the encryption IV, you then need to then make sure you
> re-encrypt any files that move between these. I am hesitant to do that
> since it then requires these workarounds for encryption going forward.
> We know that most people will not be using encryption, so that will not
> be well tested either. For pg_upgrade, I used a minimal-impact
> approach, and it has allowed dramatic changes in our code without
> requiring changes and retesting of pg_upgrade.
>

Will there be a per-relation salt stored in a separate file? I saw it
mentioned in a few places (most recently
https://www.postgresql.org/message-id/aa386c3f-fb89-60af-c7a3-9263a633ca1a%40postgresql.org)
but there's also discussion of trying to make the TDEK unique without a
separate salt so I'm unsure.

With a per-relation salt there is no need to include fixed attributes
(database, relfilenode, or tablespace) to ensure the derived key is unique
per relation. A long salt (32-bytes from /dev/urandom) alone guarantees
that uniqueness. Copying or moving files would then be possible by also
copying the salt. It does not need to be a salt per file on disk either,
one salt can be used for many files for the same relation by including the
fork number, type, or segment in the TDEK derivation (so each file on disk
for that relation ends up with a unique TDEK).

There's the usual gotchas of copying encrypted data, i.e. it's exactly the
same so clearly they're equal. But any subsequent changes would have a
different LSN and encrypt differently going forward. If the main use cases
are copying an entire database or moving a tablespace, having that be
simpler/faster seems like a good idea. It could be a known limitation like
the promoting of multiple replicas. Plus with a key rotation tool anyone
that wants everything re-encrypted could run one after the copy.

Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/