Re: Key management with tests

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Stephen Frost <sfrost(at)snowman(dot)net>
Cc: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Masahiko Sawada <masahiko(dot)sawada(at)2ndquadrant(dot)com>
Subject: Re: Key management with tests
Date: 2021-01-08 21:17:09
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Jan 8, 2021 at 03:33:44PM -0500, Stephen Frost wrote:
> > No, I don't think so. Stephen imported the entire NIST test suite. It
> > was so comperhensive, it detected several OpenSSL bugs for zero-length
> > strings, which I already reported, but we would never be encrypting
> > zero-length strings, so there wasn't a lot of value to it.
> I ran the entire test suite locally to ensure everything worked, but I
> didn't actually include all of it in the PR which you merged- I had
> already reduced it quite a bit by removing all 'additional
> authenticated data' test cases (which the tests will automatically skip
> and which we haven't implemented support for in the common library
> wrappers) and by removing the 192-bit cases. This reduced the overall
> test set by about 2/3rd's or so, as I recall.

Wow, so that was reduced!

> > Anyway, I think we need to figure out how to trim. The first part would
> > be to figure out whether we need 128 _and_ 256-bit tests, and then see
> > what items are really useful. Stephen, do you have any ideas on that?
> > We currently have 10296 tests, and I think we could get away with 100.
> Yeah, it's probably still too much, but I don't have any particularly
> justifiable suggestions as to exactly what we should remove or what we
> should keep.
> Perhaps it'd make sense to try and cover the cases that are more likely
> to be issues between our wrapper functions and OpenSSL, and not stress
> too much about constantly testing cases that should really be up to
> OpenSSL. As such, I'd propose:
> - Add back in some 192-bit tests, so we cover all three bit lengths.
> - Add back in some additional authenticated test cases, just to make
> sure that, until/unless we implement support, the test code properly
> skips over those.
> - Keep tests for various length plaintext/ciphertext (including 0-byte
> cases, so we make sure those work, since they really should).
> - Keep at least one test for each length of tag that's included in the
> test suite.

Makes sense. I did a simplistic trim-down to 90 tests but it still was
40% of the patch; attached. The hex strings are very long.

> I'm not sure how many tests we'd end up with from that, but my swag /
> gut feeling is that it'd probably be on the order of 100ish and a small
> enough set that it won't dwarf the rest of the patch.
> Would be nice if we had a way for some buildfarm animal or something to
> pull in the entire suite and test it, imv.. If anyone wants to
> volunteer, I'd be happy to explain how to make that happen (it's not
> hard though- download/unzip the files, drop them in the directory,
> update the test script to add all the files into the array).

Yes, do we have a place to store more comprehensive tests outside of our
git tree? Has this been done before?

Bruce Momjian <bruce(at)momjian(dot)us>

The usefulness of a cup is in its emptiness, Bruce Lee

Attachment Content-Type Size
cryptotest.tgz application/x-gtar-compressed 73.7 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2021-01-08 21:18:59 Re: Key management with tests
Previous Message Stephen Frost 2021-01-08 20:34:23 Re: Key management with tests