Real Case Application of Coverage Tests on Linux - Part 3

These special topics on coverage tests I made are based on real case situations of tests implementation, when I used to think about a nice way to certify that the user requirements needs related to a MythTV protocol API implementation I´m developing are well concerned. This API is not the official one, based on the MythTV server and client implementations, but another totally different effort based on GObject/GLib frameworks and object model.

The main test architecture I had developed is to allow that the test script's writer could develops the testing scripts with the minimum possible effort. So, I choose to use Python as the language of choice on the test scripts, because it's easier to learn, and removes the extra step of compiling code, because Python is interpreted. Each one of these Python scripts do, at least, one basic task: start a process with the testing program, compiled with the gcov program line arguments. A very simple strategy had been used to guarantee that the process testing stoped at a given known time, and the Python script could be signalized about the program termination.

You can see all these concepts, and see how to code gcov-based tests in practice, seeing my contribution on GMyth's SourceForge: http://sourceforge.net/projects/gmyth.

Have a lot of fun! :)

More on Coverage Tests on Linux - Generating Better Reports

At the last post, I showed up some code that could be interesting for testing purposes, and to see how the gcov do these automatic testing procedures. In fact, gcov uses a number of techniques to cover a given test code. Internally, we can say that gcov will insert into your program object file, some kind of meta-code, which will try for a large number of meaningful execution paths. So, it uses some know strategies to select what are the most effective paths he must to cover, because is theoretically impossible just try to cover all the possible situations when a program is running.

There are 2 main types of coverage: branch coverage and loop coverage. Just to give you an idea, in the branch coverage, it is important to verify that every branch has been taken. When it uses the loop coverage, gcov try to take all the execution paths through a loop. To see more about gcov and some information about some internal working issues, refer to http://gcc.gnu.org/onlinedocs/gcc/Gcov.html.

Since our gcov execution at the last article produced some files that encapsulates all the information above, we'd got almost all we need. But could be better if we could put all the data collected from gcov in a more beautiful and useful way. That is what a tool called lcov do:

lcov -d . -c -o test_gcov.info

This simple command will search for the files generated by gcov (*.gcda, *.gcno) at the current (".") directory (-d option), and put this in the test_gcov.info. This file has the collected information about the program execution and automatic testing, but is not yet in a human-readable form. But we can put this file as an input to a very useful tool, genhtml:

genhtml test_gcov.info

After this program script finishes execution, type "ls" on your prompt and see what it had just created: a lot of images (.png), CSS (cascading style sheets) and HTML files. Open the "index.html" in your preferable browser, and it will show you how powerful this tool can really be, as a valuable way to certify that your test scripts are covering the software requirements initially specified, which test cases are important, and if you can find some dead code that you can throw off.