Re: Key management with tests

Greetings,

* Bruce Momjian (bruce(at)momjian(dot)us) wrote:
> On Fri, Jan 8, 2021 at 03:33:44PM -0500, Stephen Frost wrote:
> > > Anyway, I think we need to figure out how to trim. The first part would
> > > be to figure out whether we need 128 _and_ 256-bit tests, and then see
> > > what items are really useful. Stephen, do you have any ideas on that?
> > > We currently have 10296 tests, and I think we could get away with 100.
> >
> > Yeah, it's probably still too much, but I don't have any particularly
> > justifiable suggestions as to exactly what we should remove or what we
> > should keep.
> >
> > Perhaps it'd make sense to try and cover the cases that are more likely
> > to be issues between our wrapper functions and OpenSSL, and not stress
> > too much about constantly testing cases that should really be up to
> > OpenSSL. As such, I'd propose:
> >
> > - Add back in some 192-bit tests, so we cover all three bit lengths.
> > - Add back in some additional authenticated test cases, just to make
> > sure that, until/unless we implement support, the test code properly
> > skips over those.
> > - Keep tests for various length plaintext/ciphertext (including 0-byte
> > cases, so we make sure those work, since they really should).
> > - Keep at least one test for each length of tag that's included in the
> > test suite.
>
> Makes sense. I did a simplistic trim-down to 90 tests but it still was
> 40% of the patch; attached. The hex strings are very long.

I don't think we actually need to stress over the size of the test data
relative to the size of the patch- it's not like it's all that much perl
code. I can appreciate that we don't want to add megabytes worth of
test data to the git repo though.

> > I'm not sure how many tests we'd end up with from that, but my swag /
> > gut feeling is that it'd probably be on the order of 100ish and a small
> > enough set that it won't dwarf the rest of the patch.
> >
> > Would be nice if we had a way for some buildfarm animal or something to
> > pull in the entire suite and test it, imv.. If anyone wants to
> > volunteer, I'd be happy to explain how to make that happen (it's not
> > hard though- download/unzip the files, drop them in the directory,
> > update the test script to add all the files into the array).
>
> Yes, do we have a place to store more comprehensive tests outside of our
> git tree? Has this been done before?

Not that I'm aware of.

Thanks,

Stephen