Re: Thoughts on unit testing?

On 14 August 2017 at 03:19, Peter Geoghegan <pg(at)bowt(dot)ie> wrote:

> On Thu, Aug 10, 2017 at 2:53 PM, Thomas Munro
> <thomas(dot)munro(at)enterprisedb(dot)com> wrote:
> > The current regression tests, isolation tests and TAP tests are very
> > good (though I admit my experience with TAP is limited), but IMHO we
> > are lacking support for C-level unit testing. Complicated, fiddly
> > things with many states, interactions, edge cases etc can be hard to
> > get full test coverage on from the outside. Consider
> > src/backend/utils/mmgr/freepage.c as a case in point.
>
> It is my understanding that much of the benefit of unit testing comes
> from maintainability. It's something that goes well with design by
> contract. Writing unit tests is a forcing function. It requires
> testable units, making the units more composable. The programmer must
> be very deliberate about state, and must delineate code as testable
> units. Unit tests are often supposed to be minimal, to serve as a kind
> of design document.

This is why I personally only find true unit tests useful as part of large
software packages like Pg when they cover functional units that can be
fairly well isolated.

However, I'm not sure that Thomas meant unit tests in the formal sense
isolated and mocked, but rather in the sense of test procedures written in
C to excercise portions of Pg's code that are not easily/conveniently
tested from SQL.

Pg lacks the strict module delineation needed to make formal unit testing
practical, as you have noted. That doesn't mean test frameworks couldn't be
useful. I routinely write tests for reasonably isolated functionality in my
code by writing throwaway SQL-callable functions to exercise it and
pg_regress tests to run them. It's not strict unit testing as the rest of
Pg's APIs aren't mocked away, but it's very practical small-unit
integration testing that helps catch issues.

I wouldn't mind having an easier and nicer way to do that built in to Pg,
but don't have many ideas about practical, low-maintenance ways to achieve
it.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services