Re: Proposed patch for key managment

Hi Bruce et al

Firstly, thanks for shaping the patch, getting it down to a manageable
scope of cluster file encryption. I think this is a great feature and it
matters to a lot of the customers I talk to at VMware about
adopting Postgres.

Since it's exciting stuff, I've been trying to lash together a PoC
integration with the crypto infrastructure we see at these customers.
Unfortunately, in short, the API doesn't seem to suit integration with HSM
big iron, like Thales, Utimaco, (other options are available), or their
cloudy lookalikes.

I understand the model of wrapping the Data Encryption Key and storing the
wrapped key with the encrypted data. The thing I can't find support for in
your patch is passing a wrapped DEK to an external system for decryption. A
key feature of these key management approaches is that the
application handling the encrypted data doesn't get the KEK, the HSM/KSM
passes the DEK back after unwrapping it. Google have a neat illustration of
their approach, which is very similar to others, at
https://cloud.google.com/kms/docs/envelope-encryption#how_to_decrypt_data_using_envelope_encryption

So instead of calling out to a passphrase command which returns input for a
PBKDF, Postgres (in the form of initdb or postmaster) should be passing the
wrapped DEK and getting back the DEK in plain. The value passed from
Postgres to the external process doesn't even have to be a wrapped DEK, it
could be a key in the key->value sense, for use in a lookup against Vault,
CredHub or the Kubernetes secret store. Making the stored keys opaque to
the Postgres processes and pushing the task of turning them into
cryptographic material to an external hepler process probably wouldn't
shrink the patch overall, but it would move a lot of code from core
processes into the helper. Maybe that helper should live in contrib, as
discussed below, where it would hopefully be joined by a bridge for KMIP
and others.

On Tue, 15 Dec 2020 at 19:09, Stephen Frost <sfrost(at)snowman(dot)net> wrote:

> Greetings,
>
> * Bruce Momjian (bruce(at)momjian(dot)us) wrote:
> > On Tue, Dec 15, 2020 at 10:05:49AM +0900, Michael Paquier wrote:
> > > On Mon, Dec 14, 2020 at 06:06:15PM -0500, Bruce Momjian wrote:

<snip>

> > > > There are a few shell script I should include to show how to create
> > > > commands. Where should they be stored? /contrib module?
> > >
> > > Why are these needed. Is it a matter of documentation?
> >
> > I have posted some of the scripts. They involved complex shell
> > scripting that I doubt the average user can do. The scripts are for
> > prompting for a passphrase from the user's terminal, or using the
> > Yubikey, with our without typing a pin. I can put them in the docs and
> > then users can copy them into a file. Is that the preferred method?
>
> If we are going to include these in core anywhere, I would think a new
> directory under contrib would make the most sense. I'd hope that we
> could find a way to automate the testing of them though, so that we have
> some idea when they break because otherwise I'd be concerned that
> they'll break somewhere down the line and we won't notice for quite a
> while.
>
> Regards

Alastair