Generating code coverage reports

I have a patch that I will be submitting to add to the build system the
capability of reporting on test code coverage metrics for the test
suite. Actually it can show coverage for any application run against
PostgreSQL. Download Image:Coverage.tar.gz
<http://wiki.postgresql.org/wiki/Image:Coverage.tar.gz> to see an
example report. Gunzip and un-tar the file and click on
coverage/index.html. I had to delete most of the files to decrease the
file size for upload, so only the links for access work.

gcov reports line, branch, and function coverage, but lcov only reports
on line coverage. I've added a link to the html to the gcov output that
shows summary statistics for each file for line, branch, and function
calls.

The report gives a very clear and browseable view of what parts of the
system might benefit from more extensive testing. It's obviously useful
for planning future testing, but also can be used in conjunction with
debugging to see what lines and functions are being exercised or missed
by existing tests of the functionality under investigation. It could
even be helpful to give a static view of lines hit by a bug test case in
lieue of using a debugger. Also, when you're writing a unit test for new
functionality, it would be good to check what you're actually hitting
with the test.

It uses gcov together with gcc to generate the statistics, and the lcov
suite to create the html report. Both of these would obviously have to
be installed to get a coverage report, but this would be an optional
feature of the build. It only works with gcc.

To generate coverage statistics, you run configure with
--enable-coverage and after building and running tests, you do make
coverage. The process generates data files in the same directories as
source & object files and produces a coverage directory at the top level
with the html files. I've also set it up so a tar file with the html is
generated.

More information on gcov at http://gcc.gnu.org/onlinedocs/gcc/Gcov.html,
lcov at http://ltp.sourceforge.net/documentation/how-to/ltp.php
(coverage/lcov tabs).

-- Michelle

--
Michelle Caisse Sun Microsystems
California, U.S. http://sun.com/postgresql