Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019 at 10:44 PM Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>
> On Mon, Jul 29, 2019 at 08:43:06PM +0900, Masahiko Sawada wrote:
> > > I am thinking of writing some Assert() code that checks that all buffers
> > > using a single LSN are from the same relation (and therefore different
> > > page numbers). I would do it by creating a static array, clearing it on
> > > XLogBeginInsert(), adding to it for each XLogInsert(), then checking on
> > > PageSetLSN() that everything in the array is from the same file. Does
> > > that make sense?
> >
> > I had the same concern before. We could have BKPBLOCK_SAME_REL flag in
> > XLogRecordBlockHeader, which indicates that the relation of the block
> > is the same as the previous block and therefore we skip to write
> > RelFileNode. At first glance I thought it's possible that one WAL
> > record can contain different RelFileNodes but I didn't find any code
> > attempting to do that.
>
> Yes, the point is that the WAL record makes it possible, so we either
> have to test for it or allow it.
>
> > Checking that all buffers using a single LSN are from the same
> > relation would be a good idea but I think it's hard to test it and
> > regard the test result as okay. Even if we passed 'make checkworld',
> > it might still be possible to happen. And even assertion failures
>
> Yes, the problem is that if you embed the relfilenode or tablespace or
> database in the encryption IV, you then need to then make sure you
> re-encrypt any files that move between these. I am hesitant to do that
> since it then requires these workarounds for encryption going forward.
> We know that most people will not be using encryption, so that will not
> be well tested either. For pg_upgrade, I used a minimal-impact
> approach, and it has allowed dramatic changes in our code without
> requiring changes and retesting of pg_upgrade.
>
> > don't happen in production environment. So I guess it would be better
> > to have IV so that we never reuse in different relation with the same
> > page. An idea I came up with is that we make IV from (PageLSN,
> > PageNumber, relNode) and have the encryption keys per tablespace.
> > That way, we never reuse IV in a different relation with the same page
> > number because relNode is unique within a database in a particular
> > tablespace as you mentioned.
>
> Yes, this is what we are discussing. Whether the relfilenode is part of
> the IV, or we derive a key with a mix of the master encryption key and
> relfilenode is mostly a matter of what fits into which bits. With CTR,
> I think we agreed it has to be LSN and page-number (and CTR counter),
> and we only have 5 bits left. If we wanted to add anything else, it
> would be done via the creation of a derived key; this was covered here:
>

Just to confirm, we have 21 bits left for nonce in CTR? We have LSN (8
bytes), page-number (4 bytes) and counter (11 bits) in 16 bytes nonce
space. Even though we have 21 bits left we cannot store relfilenode to
the IV.

BTW I've received a review about the current design by some
cryptologists in our company. They recommended to use CTR rather than
CBC. The main reason is that in block cipher it's important to ensure
the uniqueness even for every input block to the block cipher. CBC is
hard to ensure that because the previous output is the next block's
input. Whereas in CTR, it encrypts each blocks separately with
key+nonce. Therefore if we can ensure the uniqueness of IV we can meet
that. Also it's not necessary to encrypt IV as it's okey to be
predictable. So I vote for CTR for at least for tables/indexes
encryption, there already might be consensus though.

For WAL encryption, before flushing WAL we encrypt whole 8k WAL page
and then write only the encrypted data of the new WAL record using
pg_pwrite() rather than write whole encrypted page. So each time we
encrypt 8k WAL page we end up with encrypting different data with the
same key+nonce but since we don't write to the disk other than space
where we actually wrote WAL records it's not a problem. Is that right?

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center