Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 3/7/19 10:06, Robert Haas wrote:
> Generally, I think our interest should be less in how secrets are
> stored inside the database than in how we can integrate with an
> external secrets-management solution, and I think that's what Jeremy
> is talking about here. I don't know exactly how that would work, but
> you can imagine having a way to tell an FDW "hey, there's a password
> for this server, but it's not stored here -- instead go fetch secret
> d41d8cd98f00b204e9800998ecf8427e" and the server does that and uses
> that password for the connection. But we don't need to solve the FDW
> problem for this effort to move forward. We do, however, need a
> solution that's good enough for whatever we want to do in terms of
> TDE.

Right: the biggest use case I'm thinking about is external secret
management systems. For anyone with heavy-weight security requirements,
this will be a must.  I use LastPass in my personal life and they have
an enterprise product with API access [1] that I know nothing about. 
:)  At one previous company where I worked, they used Thycotic [2] which
is now OEMed as IBM Security Server [3]. HashiCorp Vault [4] is pretty
widely known, and it's docs have a handy list of a whole bunch more
Secrets Engines [5] they integrate with.  Every major cloud provider has
a secrets solution (AWS [6], Azure [7], GCP [8], etc) and then there are
open source secret management suites like Cerberus [9] that layer on top
of cloud APIs.  There are the services built into orchestration
frameworks like Docker [10] and k8s [11].  And of course, don't forget
HSMs [12].

I'm in no way discrediting a full implementation within an extension as
you've done, Bruce, with pgcryptokey.  In fact I think we will need
something like this as a reference implementation, and to build unit
tests.  But the problem is that this doesn't provide a standard API for
extensions to code against.  Other extensions need a dependency on
pgcryptokey, it's up to each extension author to support every secret
provider, and realistically FDWs and logical rep can't ever use an API
that's not in core.  In my ideal world, core gives us a standard API
that internal code and extensions can each code against to (1)
store/retrieve secrets [including temporary secrets or tokens;
supporting models like kerberos] and (2) provide custom backend
implementations to that service.

In short, core could provide the _plumbing_ of a standard secrets API
and allow extensions to register as providers and act as consumers of
the API.  FDWs, logical replication and TDE are the things on the table
right now but there are lots of conceivable things that future
developers might need secrets for.  TDE is a great excuse to get an API
in place. If it proves successful, then later on we can look at updating
logical replication and FDWs to use this API as well.

-Jeremy

1. https://www.lastpass.com/enterprise-password-management
2. https://thycotic.com/products/secret-server/
3.
https://ovum.informa.com/resources/product-content/ibm-adopts-thycotic-for-privileged-account-management
4. https://www.vaultproject.io/
5. https://www.vaultproject.io/docs/secrets/index.html
6. https://aws.amazon.com/secrets-manager/
7. https://azure.microsoft.com/en-us/services/key-vault/
8. https://cloud.google.com/solutions/secrets-management/
9. http://engineering.nike.com/cerberus/docs/
10. https://docs.docker.com/engine/swarm/secrets/
11. https://kubernetes.io/docs/concepts/configuration/secret/
12. https://en.wikipedia.org/wiki/Hardware_security_module

--
Jeremy Schneider
Database Engineer
Amazon Web Services