Rostest not showing existing failures and errors

asked 2019-05-25 12:10:45 -0500

minennick gravatar image

Hey guys, i have a problem with rostest not showing existing failures, but only succeding tests. I'm working on Kinetic and ROS 1.12.14

For this testfile:

#!/usr/bin/env python
PKG = 'bobtimus_lane_detection'

import sys
import unittest
import rostest


## A sample python unit test
class TestCareBones(unittest.TestCase):
    ## test 1 == 1
    def test_one_equals_one(self): # here's the failure
        self.assertTrue(1 == 2)
    def test_two_equals_two(self):
        self.assertEqual(2, 2, "2!=2")


if __name__ == '__main__':
    rostest.rosrun(PKG, 'test_care_bones', TestCareBones)

and this simple launchfile:

<launch>
  <test test-name="test_lane_detection" pkg="bobtimus_lane_detection" type="lane_detection_test.py" />
</launch>

i get this output:

[ROSUNIT] Outputting test results to /home/niklas/.ros/test_results/bobtimus_lane_detection/rostest- 
 launch_lane_detection.xml
[Testcase: testtest_lane_detection] ... ok

[ROSTEST]-----------------------------------------------------------------------


SUMMARY
 * RESULT: SUCCESS
 * TESTS: 0
 * ERRORS: 0
 * FAILURES: 0

looking into the the given results file (rostest- launch_lane_detection.xml) i can see this warning probably causing the error:

WARN: test result file is empty [/home/niklas/.ros/test_results/bobtimus_lane_detection/rosunit-test_lane_detection.xml]

If i correct the false test above everything works fine and the results get written in the file. So why is this happening and how can it be fixed? Thanks for your help :)

edit retag flag offensive close merge delete

Comments

Hi,

I am observing a similar behavior. Have you been able to identify a solution for this problem?

System being used: ROS Melodic

Thank.

rktb gravatar image rktb  ( 2019-10-18 13:19:12 -0500 )edit

I have the same issue, have not solved it yet but just incase in the xml file you are watching look for another xml file ("results are"), if there is, open that file and you will see the error you are expecting. For some reason the Summary is success though

Kansai gravatar image Kansai  ( 2021-02-16 22:50:38 -0500 )edit

Same issue here. Running both catkin runtests and rostest both says SUCCESS but looking at the rosunit xml result shows that there are failed asserts.

ROS version: Melodic

frpunzalan gravatar image frpunzalan  ( 2021-02-17 01:12:48 -0500 )edit

I just can't repro this issue using your code and ran it by rostest bobtimus_lane_detection foo.test, but on Ubuntu 16.04, Kinetic. I vaguely remember there was a regression either in Ubuntu 18.04 or newer in the upstream (log4j or something else) so that the test format broke but in ROS it got patched immediately. I just can't find it but if that's correct then that might be affecting.

130s gravatar image 130s  ( 2021-03-16 23:49:28 -0500 )edit