Using CMake, GoogleTests and gcovr in a C project

Date Mon 06 January 2025 By Emmanuel Fleury Category programming Tags C / CMake / GoogleTests / gcovr

This article presents a simple way to setup a C project using CMake, GoogleTest and gcovr. Setting all together and making them work can be a bit tricky, so I will try to explain the steps to make it work.

Project Hierarchy

Our project will only contain one module and one function for the sake of simplicity. The hierarchy of the project will be as follows:

hello.git/
├── CMakeLists.txt
├── LICENSE.md
├── README.md
├── include/
│   └── utils.h
├── src/
|   ├── CMakeLists.txt
│   ├── hello.c
│   └── utils.c
└── tests/
    ├── CMakeLists.txt
    ├── hello_test.c
    └── utils_test.c

Our main program will be in src/hello.c and the corresponding test in tests/hello_test.c. The utils.c and utils.h files will contain some utility functions that will be tested in tests/utils_test.c.

The Code

We provide two simple functions in utils.c and utils.h: max() and print_hello(). Then, hello.c is the main program that run on console.

The content of hello.c is the following:

#include <stdio.h>
#include <stdlib.h>

#include <utils.h>

int main (int argc, char *argv[]) {
  if (argc == 1) {
    printf ("Hello World!\n");
    return EXIT_SUCCESS;
  }

  print_hello (atoi (argv[1]));

  return EXIT_SUCCESS;
}

The content of utils.c is the following:

#include "utils.h"

#include <stdio.h>

int max (int a, int b)
{
  if (a > b)
    return a;
  return b;
}

void print_hello (int count)
{
  for (int i = 0; i < count; i++)
    printf ("Hello World!\n");
}

And, the content of utils.h is the following:

#ifndef UTILS_H
#define UTILS_H

/* Returns the max of 'a' and 'b' */
int max (int a, int b);

/* Prints "Hello, world!" 'count' times on stdout */
void print_hello (int count);

#endif

Build-System (CMake)

Set up the CMakeLists.txt files requires to have a CMakeLists.txt file in the root directory of the project and one in src/ in charge of building the main program.

The main CMakeLists.txt file is the following:

# Minimum version of CMake required to build this project
cmake_minimum_required(VERSION 3.15)

# Name of the project
project(hello LANGUAGES C)

# Set the C standard to C11
set(CMAKE_C_STANDARD 11)

# Add subdirectories
add_subdirectory(src)

The src/CMakeLists.txt file is the following:

# Include directories
include_directories(${CMAKE_SOURCE_DIR}/include)

# Move executables to the root of the build directory
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})

# Add target and source files
add_executable(hello hello.c utils.c)

Now you can build the project by running the following commands:

#> mkdir build
#> cd build
#> cmake ..
#> make

The point of having the binary files in a build/ directory is to keep the source directory clean. It also allows to manage several build directories for different configurations (e.g., debug, release, etc.). CMake has several default build types such as Debug and Release. You can specify the build type by running cmake -DCMAKE_BUILD_TYPE=Debug .. or cmake -DCMAKE_BUILD_TYPE=Release ... So, for example, you could do:

#> mkdir release
#> cd release
#> cmake -DCMAKE_BUILD_TYPE=Release ..
#> make
#> cd ../
#> mkdir debug
#> cd debug
#> cmake -DCMAKE_BUILD_TYPE=Debug ..
#> make

Thus, you can have two different build directories for a build in debug mode and another in release mode.

Finally, note also that we usually create the build/ directory at the root of the sources but it could be anywhere else.

Testing (GoogleTest)

Lets now add the GoogleTest framework to our project. We will use CMake to install it locally to our project and avoid any system-wide installation.

One may notice that the GoogleTest framework is in C++ and we are using C. This is not a problem because we can easily mix C and C++ code in the test files. Yet, this requires to have a C++ compiler installed on the system and to learn a bit C++.

Lets first get the GoogleTest framework installed in our project through CMake (to be added in the main CMakeLists.txt file):

...
# Name of the project
project(hello LANGUAGES C CXX) # We require C++ for GoogleTest

# Set the C standard to C11
set(CMAKE_C_STANDARD 11)

# Set the C++ standard to C++17
set(CMAKE_CXX_STANDARD 17)

# Fetch GoogleTest framework and build it
include(FetchContent)
FetchContent_Declare(
        googletest
        URL https://github.com/google/googletest/releases/download/v1.15.2/googletest-1.15.2.tar.gz
        PREFIX ${CMAKE_CURRENT_BINARY_DIR}/gtest
        DOWNLOAD_EXTRACT_TIMESTAMP TRUE
        DOWNLOAD_NO_PROGRESS TRUE
      )
set(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
FetchContent_MakeAvailable(googletest)
...

So, the FetchContent_Declare() function is used to download the GoogleTest from github and the FetchContent_MakeAvailable() function is used to build it.

Now, lets add some tests to our project in the tests/ directory. First, we test utils functions in tests/utils_test.c:

#include <gtest/gtest.h>

extern "C" { /* Because our code is in C */
  #include <utils.h>
}

/* Tests on utils max() function */
TEST (UtilsTest, MaxTest) {
  EXPECT_EQ (max (1, 2), 2);
  EXPECT_EQ (max (2, 1), 2);
  EXPECT_EQ (max (1, 1), 1);
}

/* Tests on utils print_hello() function */
TEST (UtilsTest, PrintHelloTest) {
  testing::internal::CaptureStdout ();
  print_hello (0);
  EXPECT_STREQ (testing::internal::GetCapturedStdout ().c_str (), "");

  testing::internal::CaptureStdout ();
  print_hello (1);
  EXPECT_STREQ (testing::internal::GetCapturedStdout ().c_str (),
                "Hello World!\n");

  testing::internal::CaptureStdout ();
  print_hello (2);
  EXPECT_STREQ (testing::internal::GetCapturedStdout ().c_str (),
                "Hello World!\nHello World!\n");

  testing::internal::CaptureStdout ();
  print_hello (-1);
  EXPECT_STREQ (testing::internal::GetCapturedStdout ().c_str (), "");
}

And, then, we test the hello program in tests/hello_test.c:

#include <gtest/gtest.h>

#define BUF_SIZE 128

/* Helper function to execute a command and capture its output */
std::string execute_command(const std::string& command) {
  std::ostringstream output;
  std::array<char, BUF_SIZE> buffer;
  FILE* pipe = popen(command.c_str(), "r");

  if (!pipe)
    throw std::runtime_error("popen() failed!");

  while (fgets(buffer.data(), buffer.size(), pipe) != nullptr)
    output << buffer.data();

  pclose(pipe);
  return output.str();
}

/* Tests on main program */
TEST (HelloTest, MainTest) {
  std::string bindir = BINDIR;
  std::string command =  bindir + std::string("/hello");
  EXPECT_STREQ (execute_command(command).c_str(), "Hello World!\n");

  command = bindir + std::string("/hello 2");
  EXPECT_STREQ(execute_command(command).c_str(), "Hello World!\nHello World!\n");

  command = bindir + std::string("/hello A");
  EXPECT_STREQ(execute_command(command).c_str(), "");
}

Here, we define an helper function execute_command() to run the hello program and check how it behaves on various inputs. Note also that to find the path to the executable hello, we use a BINDIR variable that we will define in the CMakeLists.txt file in the tests/ directory.

And, here is the content of the tests/CMakeLists.txt file:

# Include subdirectories
include_directories(${GTEST_INCLUDE_DIRS} ${CMAKE_SOURCE_DIR}/include)

# hello_tests (tests on the 'hello' binary)
## Set the binary directory as a macro
add_compile_definitions(BINDIR="${CMAKE_BINARY_DIR}")
## Add the tests
add_executable(hello_tests hello_tests.cc)
target_link_libraries(hello_tests GTest::gtest_main)
add_test(NAME hello_tests COMMAND hello_tests)

# utils_tests (tests on the 'utils' module)
## Add the tests
add_executable(utils_tests utils_tests.cc ${CMAKE_SOURCE_DIR}/src/utils.c)
target_link_libraries(utils_tests GTest::gtest_main)
add_test(NAME utils_tests COMMAND utils_tests)

A test requires to build an executable and to link it with the GoogleTest. Then, we add the test to the list of tests to run with the add_test() function.

Note that we chose to name the test and the executable the same way. This is not mandatory but it makes things simpler.

Then we need to add to the main CMakeLists.txt file the following lines:

# Tests
include(CTest)
include(GoogleTest)

enable_testing()

# Add subdirectories
add_subdirectory(src)
add_subdirectory(tests)

Now, you can run the tests by running the following commands inside the build directory:

#> ctest
#> ctest --verbose
#> make test

Code Coverage (gcovr)

It is now time to add code coverage to our project. We use the gcovr framework which is a Python module that uses the GNU gcov tool to produce code coverage reports.

The coverage is achieved by instrumenting the code with the --coverage flag when compiling and linking the program. The gcovr script is then used to generate the report.

First, we need to add a custom target to the main CMakeLists.txt file to run the gcovr script:

...
# Add a custom target for generating coverage reports
if (BUILD_TYPE STREQUAL "Coverage") # Check if the build type is 'Coverage'
  find_package(Python3 REQUIRED COMPONENTS Interpreter)  # Required for gcovr
  add_custom_target(coverage
    COMMAND ${CMAKE_COMMAND} -E make_directory coverage
    COMMAND ${Python3_EXECUTABLE} -m gcovr -r ${CMAKE_SOURCE_DIR} --html --html-nested -o coverage/index.html
    WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
    COMMENT "Generating HTML coverage report..."
  )
endif()
...

Note that we add this target only if the build type is Coverage. This is to avoid to keep the coverage instrumentation in the other build types. Coverage is only needed in development and the instrumentation embedded in the executable may slow it down.

Then, to add the compilation flag --coverage to the project, we need to modify the src/CMakeLists.txt file as follows:

# Enable coverage flags for the Coverage build type on GCC/Clang
if(CMAKE_C_COMPILER_ID MATCHES "GNU|Clang")
  set(CMAKE_C_FLAGS_COVERAGE "${CMAKE_C_FLAGS_DEBUG} --coverage")
  set(CMAKE_EXE_LINKER_FLAGS_COVERAGE "${CMAKE_EXE_LINKER_FLAGS_DEBUG} --coverage")
  set(CMAKE_SHARED_LINKER_FLAGS_COVERAGE "${CMAKE_SHARED_LINKER_FLAGS_DEBUG} --coverage")
  set(CMAKE_MODULE_LINKER_FLAGS_COVERAGE "${CMAKE_MODULE_LINKER_FLAGS_DEBUG} --coverage")
endif()

Then, to have a complete coverage report, we need to run the following commands:

#> mkdir coverage
#> cd coverage
#> cmake -DCMAKE_BUILD_TYPE=Coverage ..
#> make
#> make test
#> make coverage
#> firefox coverage/index.html

The make coverage command will generate the coverage report in the coverage/ directory. You can then open the index.html file in a web browser to see the coverage report.

Gathering all together

We modified quite a lot the CMakelists files to add GoogleTest and gcovr to our project, here is the final version of the main CMakeLists.txt file:

# Minimum version of CMake required to build this project
cmake_minimum_required(VERSION 3.15)

# Name of the project
project(hello LANGUAGES C CXX)

# Set the C standard to C11
set(CMAKE_C_STANDARD 11)

# Set the C++ standard to C++17
set(CMAKE_CXX_STANDARD 17)

# Fetch GoogleTest framework and build it
include(FetchContent)
FetchContent_Declare(
        googletest
        URL https://github.com/google/googletest/releases/download/v1.15.2/googletest-1.15.2.tar.gz
        PREFIX ${CMAKE_CURRENT_BINARY_DIR}/gtest
        DOWNLOAD_EXTRACT_TIMESTAMP TRUE
        DOWNLOAD_NO_PROGRESS TRUE
      )
set(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
FetchContent_MakeAvailable(googletest)

# Tests
include(CTest)
include(GoogleTest)

enable_testing()

# Add a custom target for generating coverage reports
if (BUILD_TYPE STREQUAL "Coverage") # Check if the build type is 'Coverage'
  find_package(Python3 REQUIRED COMPONENTS Interpreter)  # Required for gcovr
  add_custom_target(coverage
    COMMAND ${CMAKE_COMMAND} -E make_directory coverage
    COMMAND ${Python3_EXECUTABLE} -m gcovr -r ${CMAKE_SOURCE_DIR} --html --html-nested -o coverage/index.html
    WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
    COMMENT "Generating HTML coverage report..."
  )
endif()

# Add subdirectories
add_subdirectory(src)
add_subdirectory(tests)

And, here is the final version of the src/CMakeLists.txt file:

# Set the compiler flags on the Coverage build-type
if(CMAKE_C_COMPILER_ID MATCHES "GNU|Clang")
  set(CMAKE_C_FLAGS_COVERAGE "${CMAKE_C_FLAGS_DEBUG} --coverage")
  set(CMAKE_EXE_LINKER_FLAGS_COVERAGE "${CMAKE_EXE_LINKER_FLAGS_DEBUG} --coverage")
  set(CMAKE_SHARED_LINKER_FLAGS_COVERAGE "${CMAKE_SHARED_LINKER_FLAGS_DEBUG} --coverage")
  set(CMAKE_MODULE_LINKER_FLAGS_COVERAGE "${CMAKE_MODULE_LINKER_FLAGS_DEBUG} --coverage")
endif()

# Include directories
include_directories(${CMAKE_SOURCE_DIR}/include)

# Move executables to the root of the build directory
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})

# Add target and source files
add_executable(hello hello.c utils.c)

And, finally, here is the final version of the tests/CMakeLists.txt file:

# Include subdirectories
include_directories(${GTEST_INCLUDE_DIRS} ${CMAKE_SOURCE_DIR}/include)

# Set the compiler flags on the Coverage build-type
if(CMAKE_C_COMPILER_ID MATCHES "GNU|Clang")
  set(CMAKE_C_FLAGS_COVERAGE "${CMAKE_C_FLAGS_DEBUG} --coverage")
  set(CMAKE_EXE_LINKER_FLAGS_COVERAGE "${CMAKE_EXE_LINKER_FLAGS_DEBUG} --coverage")
  set(CMAKE_SHARED_LINKER_FLAGS_COVERAGE "${CMAKE_SHARED_LINKER_FLAGS_DEBUG} --coverage")
  set(CMAKE_MODULE_LINKER_FLAGS_COVERAGE "${CMAKE_MODULE_LINKER_FLAGS_DEBUG} --coverage")
endif()

# hello_tests (tests on the 'hello' binary)
## Set the binary directory as a macro
add_compile_definitions(BINDIR="${CMAKE_BINARY_DIR}")
## Add the tests
add_executable(hello_tests hello_tests.cc)
target_link_libraries(hello_tests GTest::gtest_main)
add_test(NAME hello_tests COMMAND hello_tests)

# utils_tests (tests on the 'utils' module)
## Add the tests
add_executable(utils_tests utils_tests.cc ${CMAKE_SOURCE_DIR}/src/utils.c)
target_link_libraries(utils_tests GTest::gtest_main)
add_test(NAME utils_tests COMMAND utils_tests)

Coding style with clang-format

Now, we can add a coding style check to our project using clang-format. We first have to select a base style among: LLVM, GNU, Google, Chromium, Microsoft, Mozilla and WebKit. Then, create a .clang-format file at the root of the project with the following content:

---
Language: Cpp
BasedOnStyle: GNU
...

Then, we can run the following command to conform to the style of the code:

#> clang-format -i $(find . -name '*.[ch]')

If you want to tweak the base style, you can run the following command to dump a more detailed configuration in the .clang-format file (see here for the full list of options):

#> clang-format --style=GNU --dump-config > .clang-format

Note that once the .clang-format file is created, the clang-format command will use it by default. Moreover, you can configure your editor to use it and all the other developers will have the same coding style. That is why it is considered a good practice to have a .clang-format file at the root directory of the project.

Static analysis with clang-tidy

Another useful tool to add to the project is clang-tidy. It is a static analysis tool based on the clang compiler that checks the code for potential bugs and stylistic issues. To use it, you need to have a .clang-tidy file at the root of the project with the following content (see here for the main categories of checks and here for the full list of available checks):

---
Checks: '-*,clang-analyzer-*,readability-identifier-length'
...

Basically, the Checks field is a list of checks to run. The -* check disables all checks and clang-analyzer-* enables all the checks from the clang-analyzer module.

Then, you can run the following command to check the code:

#> clang-tidy $(find . -name '*.[ch]') -- -std=c11 -Iinclude
[1/3] Processing file include/utils.h.
[2/3] Processing file src/hello.c.
[3/3] Processing file src/utils.c.
16 warnings generated.
include/utils.h:5:13: warning: parameter name 'a' is too short, expected at least 3 characters [readability-identifier-length]
    5 | int max(int a, int b);
      |             ^
include/utils.h:5:20: warning: parameter name 'b' is too short, expected at least 3 characters [readability-identifier-length]
    5 | int max(int a, int b);
      |                    ^
src/utils.c:5:13: warning: parameter name 'a' is too short, expected at least 3 characters [readability-identifier-length]
    5 | int max(int a, int b) {
      |             ^
src/utils.c:5:20: warning: parameter name 'b' is too short, expected at least 3 characters [readability-identifier-length]
    5 | int max(int a, int b) {
      |                    ^
Suppressed 12 warnings (12 in non-user code).
Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.

The ‘-- option is used to mark the start of the options passed to the compiler. Here, we pass the ‘-std=c11 option to specify the C standard and the ‘-Iinclude option to specify the include directory.

Of course, if you remove the readability-identifier-length check from the .clang-tidy file, you will not have the warnings about the parameter names being too short. And you can locally disable a check by adding a comment in the code:

int max(int a, int b) { // NOLINT(readability-identifier-length)
  ...
}

Finally, if you want to run clang-tidy with all the checks enabled, you can simply replace your .clang-tidy file with:

---
Checks: '*'
...

But, you will need to sort out the warnings and disable the ones that are not relevant to your project. This is why it is better to start with a minimal set of checks and then add the ones that are relevant to your project little by little.

Fuzzing

Another way to reduce the number of bugs inside a C code is to try to fuzz it with some advanced fuzzer such as AFL++ (American Fuzzy Lop). I will not explain here the principle of a code coverage guided fuzzer. We will just learn how to do the fuzzing through CMake in an easy way.

First of all, we need to create a build directory with specific build-options in order to instrument our binary to let AFL++ know about the execution paths taken inside our binary:

#> mkdir build-afl/
#> cd build-afl/
#> cmake -DCMAKE_C_COMPILER=afl-gcc -DCMAKE_C_FLAGS="-fsanitize=address,undefined -fno-sanitize-recover=all -g" -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address,undefined -fno-sanitize-recover=all" -DCMAKE_MODULE_LINKER_FLAGS="-fsanitize=address,undefined -fno-sanitize-recover=all" -DCMAKE_BUILD_TYPE=Debug,ASAN,UBSAN -DBUILD_SHARED_LIBS=OFF ..
#> make

Note that we also added compilation flags to enable a memory sanitizer, this helps to force a crash when a memory violation is encountered. Indeed, a fuzzer can only detect crashes, so adding the sanitizer helps us to capture more bad behaviours in our program.

Then, we need to create an input/ and an output/ directories to give to the fuzzer a collection of typical inputs that could be fitted to the stdin of our program and to collect the outputs of the fuzzer.

#> mkdir input output
#> echo -e "aaa\naaa\n" > input/sample.txt

Now, we are ready to start. We will first try to fuzz the stdin of our program. Here is the way to do it:

#> afl-fuzz -i input/ -o output/ -- ./hello

Note that you may be asked by AFL++ to set your kernel in some specific modes in order to get the best of it during the fuzzing phase. Usually, AFL++ issue quite explicit messages about it and give the needed command to execute. One that is almost always needed to execute is the following (yes, you’d better to be root on the fuzzing machine):

#> cd /sys/devices/system/cpu
#> echo performance | sudo tee cpu*/cpufreq/scaling_governor

Once all is set and that you managed to start the fuzzing, you should see the following panel:

AFL++ Status Screen

Congratulation, you got your first AFL++ session running!

For now, you just fuzz the stdin of your software. But, if you want to fuzz some input files given on the command line, you can do:

#> afl-fuzz -i input/ -o output/ -- ./hello --file @@

Here, the @@ is a placeholder for AFL++ to fit the fuzzed file and feed your program with.

Finally, you will find all the crashes encountered in the output/default/crashes/ directory. Try it on your software to see if you encounter a real bug and solve it.

Of course, AFL++ can do much more but this is another story…