Ask Your Question
0

Using colcon test --pytest-with-coverage on a CMake-based package with both Python & C++ source files

asked 2020-03-15 12:03:25 -0500

updated 2020-03-15 12:04:44 -0500

I'm trying to generate code coverage reports. I'm running these commands in a workspace that has all repos from the master ros2.repos :

$ colcon build --packages-up-to ros2trace tracetools tracetools_launch tracetools_read tracetools_test tracetools_trace --cmake-args --no-warn-unused-cli -DCMAKE_BUILD_TYPE=Debug -DINSTALL_EXAMPLES=OFF -DSECURITY=ON --ament-cmake-args -DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} --coverage" -DCMAKE_C_FLAGS="${CMAKE_C_FLAGS} --coverage"
$ colcon test --packages-select ros2trace tracetools tracetools_launch tracetools_read tracetools_test tracetools_trace --pytest-with-coverage --event-handlers console_direct+ --executor sequential

I based the above commands on the nightly_linux_coverage job's colcon build and colcon test commands (build and test), although I removed the explicit -DBUILD_TESTING=ON because one package failed (it shouldn't change anything for the packages I do want to check).

For pure Python packages (i.e. those with setup.py files), everything works. I get coverage.xml/coverage.html/etc. files in the package's build directory (build/$PACKAGE_NAME/). However, for C++ packages (using CMake) that also have Python files, I get gcov reports, but I do not get any coverage.xml/coverage.html/etc. files.

How can I do this? Maybe I'm missing something. There unfortunately does not seem to be any tutorial that covers everything for generating code coverage reports with colcon.

I'm using colcon-core 0.5.5, which does have the --pytest-with-coverage option so that the package doesn't need to explicitly declare a test dependency on pytest-cov in its setup.py, because, well, it's a CMake package, not a Python package. https://github.com/colcon/colcon-core...

edit retag flag offensive close merge delete

Comments

1

The option --pytest-with-coverage only affects Python packages (see colcon test --help and in which group the option is described).

Dirk Thomas gravatar image Dirk Thomas  ( 2020-03-15 12:12:27 -0500 )edit

How are you invoking the test in CMake? If you e.g. use ament_add_pytest_test that function doesn't have an option to enable coverage atm. It should be fairly easy to add such an option though.

Dirk Thomas gravatar image Dirk Thomas  ( 2020-03-15 12:13:41 -0500 )edit

oh right, I totally overlooked "Arguments for 'python' packages."

christophebedard gravatar image christophebedard  ( 2020-03-15 12:15:59 -0500 )edit

I'll look into adding it then.

christophebedard gravatar image christophebedard  ( 2020-03-15 12:16:31 -0500 )edit

I am indeed using ament_add_pytest_test

christophebedard gravatar image christophebedard  ( 2020-03-15 12:17:25 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
0

answered 2020-04-17 10:14:49 -0500

As noted in the comments above, it wasn't possible.

However, as of ament_cmake#226, one can enable coverage with pytest_cov for tests added with ament_add_pytest_test using the AMENT_CMAKE_PYTEST_WITH_COVERAGE option, or it can be enabled by default for a package by declaring a <test_depend> on python3-pytest-cov.

Also, as of colcon-mixin-repository#23, there is a coverage-pytest mixin (for build) that sets that option to ON.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

2 followers

Stats

Asked: 2020-03-15 12:03:25 -0500

Seen: 59 times

Last updated: Apr 17