Empty Coverage Output in `jazzy` due to `geninfo` errors

[mwcondino]

Environment

Running inside a docker container on ros:jazzy-ros-base.

Observed issue

I use colcon lcov-result to help compute coverage data for some packages in a ROS2 stack. When I migrated that stack to jazzy, I noticed that all my packages started showing zero coverage, which seemed like a regression.

Command used

colcon lcov-result --packages-select <my_cool_custom_package> --build-base <path_to_build_space> --lcov-base <path_to_build_space>/<my_cool_custom_package>/coverage/unit-cpp --lcov-config-file $(pwd)/.lcovrc --filter "*/test/*"

Contents of .lcovrc:

geninfo_auto_base=1

# Specify size of tabs
genhtml_num_spaces = 4

# Include color legend in HTML output if non-zero
genhtml_legend = 1

# Include function coverage data display
genhtml_function_coverage = 1

# Include branch coverage data display
genhtml_branch_coverage = 1

# Specify whether to capture coverage data for external source
# files
geninfo_external = 0

# Less verbose output
lcov_quiet = 1

# Specify if function coverage data should be collected and
# processed.
lcov_function_coverage = 1

# Specify if branch coverage data should be collected and
# processed.
branch_coverage = 1

## Follow symlinks
lcov_follow = 1

genhtml_no_prefix = 0

There are some additional steps I omitted before running this command (namely, making sure that the output coverage directory actually exists).

Error message

After digging in a bit, I saw these particular messages (I replaced my actual package name w/ <my_cool_custom_package>):

--- stderr: <my_cool_custom_package>                   
geninfo: WARNING: /usr/include/c++/13/bits/stl_map.h:368: unexecuted block on non-branch line with non-zero hit count.  Use "geninfo --rc geninfo_unexecuted_blocks=1 to set count to zero.
geninfo: ERROR: "/usr/include/c++/13/bits/vector.tcc":469: mismatched exception tag for id 1, 1: '0' -> '1'
	(use "geninfo --ignore-errors mismatch ..." to bypass this error)
---
Finished <<< <my_cool_custom_package> [1.44s]

Summary: 1 package finished [1.79s]
  1 package had stderr output: <my_cool_custom_package>

Calculating total coverage... 

Applying filters... 
Summary coverage rate:
  lines......: 0.0% (0 of 24886 lines)
  functions..: 0.0% (0 of 10798 functions)
  branches...: no data found

Generating HTML: Overall coverage rate:
  lines......: 0.0% (0 of 24886 lines)
  functions......: 0.0% (0 of 10582 functions)
  branches......: no data found
Done

Existing workaround

I submitted this PR to enable a workaround. Namely, adding --lcov-args --ignore-errors mismatch seems to work around the underlying issue.

[christophebedard]

Thanks for documenting this. I've also been seeing some cryptic errors lately, but haven't had time to really investigate. Any PR that addresses the root cause would be welcome 😁

I did find this: linux-test-project/lcov#209 (comment)

[mwcondino]

Yeah if I find anything more substantive, I'm happy to post a PR.

As far as I can tell, the change from iron to jazzy caused some differences in lcov, namely that lcov=2.0 is now being used, whereas I think I was on 1.xxx before. I think the change to V2 is significant for a few reasons, but the biggest ones I'm seeing are that certain warnings before are now considered errors, hence the ERROR message

Edit: To be clear, the underlying difference has more to do with the Ubuntu version than with the ROS version, I don't think my original comment was precise. I think that Ubuntu24, used in ros docker images, has numerous upgraded packages, including lcov