Ask Your Question
1

Using rostest with gtest value-parameterized tests

asked 2011-08-05 05:17:09 -0500

jatowler gravatar image

updated 2011-08-14 16:57:23 -0500

kwc gravatar image

I have a test suite that implements Google's value-parameterized tests using a TEST_P test type and INSTANTIATE_TEST_CASE_P to define a series of tests.

When I run rostest on my .test file, only the tests defined with TEST_F are run, not the parameterized ones. However, when I run rostest --text with the same .test file, all tests, including those defined with TEST_P, are run.

What is causing the difference, and how can I cause the standard rostest call to run all of my tests and record the results to the standard XML file?

EDIT: It turns out that in fact all of the tests are being run and recorded in the XML files. However, the results are not aggregated and output to the console unless --text is specified. This is very bad, because even deliberately failing tests (with ASSERT_TRUE(false)) does not cause make test or rostest to report failure. As a result, any number of tests would fail, and I would never know unless I manually checked the XML, which is counter to the entire point of gtest.

Is this expected behavior? Or a bug I should file?

EDIT 2: Apparently, I need more karma to attach a file, so I have uploaded a complete test package to my website instead (it's a complete package, in case I mangled some configuration to cause this error). Note that make test returns success, as does rostest launch/sample.test, while the XML file gets written correctly. Running the executable itself, with ./sampletest, also correctly reports the failure. Additionally, I found that, if there are only TEST_P cases in the test file, they get output to the console as expected.

edit retag flag offensive close merge delete

Comments

I ran the rostest sample and I get test failures. Output here: http://pastebin.com/vyPZpzHi
kwc gravatar image kwc  ( 2011-08-09 06:11:16 -0500 )edit

I ran into a similar issue with the standard node-level-integration tests.

Jon Stephan gravatar image Jon Stephan  ( 2012-10-21 06:29:24 -0500 )edit

I'm seeing this same problems. I have an integration test, which have some regular TEST() tests and also some parametrized tests (declaring the class and then using TEST_P). The parametrized tests get ignored when running rostest. I don't see them in the rosunit xml either.

tulku gravatar image tulku  ( 2014-04-11 05:23:20 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
0

answered 2011-08-06 08:25:19 -0500

kwc gravatar image

rostest parses the recorded results in the XML file to determine whether or not the test fails. The aggregation and output of the summary is also produced from these XML results. The fact that the test results are not being picked up implies that the results in the XML file are not parsed correctly, or the XML file is being output in an unexpected location. In either case, it would be useful to have a small sample case to try and reproduce with.

edit flag offensive delete link more

Comments

marking as closed per comment on test failure above
kwc gravatar image kwc  ( 2011-09-02 11:24:25 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2011-08-05 05:17:09 -0500

Seen: 974 times

Last updated: Aug 08 '11