Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 08, 2019 at 05:41:55PM -0400, Bruce Momjian wrote:
>On Mon, Jul 8, 2019 at 09:30:03PM +0200, Tomas Vondra wrote:
>> I think Bruce's proposal was to minimize the time the key is "unlocked"
>> in memory by only unlocking them when the user connects and supplies
>> some sort of secret (passphrase), and remove them from memory when the
>> user disconnects. So there's no way for the auxiliary processes to gain
>> access to those keys, because only the user knows the secret.
>
>I mentioned that because I thought that was the security value that
>people wanted. While I can see the value, I don't see how it can be
>cleanly accomplished. Keeping the keys unlocked at all times seems to
>be possible, but of much smaller value.
>
>Part of my goal in this discussion is to reverse the rush to implement
>and pick apart exactly what is possible, and desirable.
>
>> FWIW I have doubts this scheme actually measurably improves privacy in
>> practice, because most busy applications will end up having the keys in
>> the memory all the time anyway.
>
>Yep.
>
>> It also assumes memory is unsafe, i.e. bad actors can read it, and
>> that's probably a valid concern (root access, vulnerabilities etc.). But
>> in that case we already have plenty of issues with data in flight
>> anyway, and I doubt TDE is an answer to that.
>
>Agreed.
>
>> > Ideally, all of this would leverage a vaulting system or other mechanism
>> > which manages access to the keys and allows their usage to be limited.
>> > That's been generally accepted as a good way to bridge the gap between
>> > having to ask users every time for a key and having keys stored
>> > long-term in memory.
>>
>> Right. I agree with this.
>>
>> > Having *only* the keys for the data which the
>> > currently connected user is allowed to access would certainly be a great
>> > initial capability, even if system processes (including potentially WAL
>> > replay) have to have access to all of the keys. And yes, shared buffers
>> > being unencrypted and accessible by every backend continues to be an
>> > issue- it'd be great to improve on that situation too. I don't think
>> > having everything encrypted in shared buffers is likely the solution,
>> > rather, segregating it up might make more sense, again, along similar
>> > lines to keys and using metadata that's outside of the catalogs, which
>> > has been discussed previously, though I don't think anyone's actively
>> > working on it.
>> >
>>
>> I very much doubt TDE is a solution to this. Essentially, TDE is a good
>> data-at-rest solution, but this seems more like protecting data during
>> execution. And in that case I think we may need an entirely different
>> encryption scheme.
>
>I thought client-level encryption or pgcrypto-style encryption fits that
>need better.
>

I'm not sure client-level encryption is something people really want.

It essentially means moving a lot of the logic to the client. For
example, when you want to do grouping on joins on encrypted columns, we
can't do that in the database when only the client knows the key. And we
know how terribly half-baked those client-side implementations are, even
without the additional encryption complexity.

So it's more a case of not having a better in-database solution :-(

pgcrypto is a bit ugly and tedious, but in general it's a step in the
right direction. The main issue is (a) the key management (or lack of
it) and (2) essentially decrypting once and then processing plaintext in
the rest of the query.

I think (1) could be improved if we had a key vault of some sorts (which
we might get as part of TDE) and (2) might be improved by having special
encrypted data types, with operations offloaded to a trusted execution
environment (TrustZone, SGX, ...).

But that's a very separate topic, unrelated to TDE.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services