The camera_info_manager package mixes gtest and rostest wrongly. Currently it specifies two tests: a plain gtest (catkin_add_gtest) and a rostest (add_rostest).
The rostest invocation should work - try to call make run_tests_camera_info_manager_rostest_.... What fails is the gtest you have declared in CMakeLists.txt. It expects it to be a plain gtest file which does not require any ROS environment.
Since unit_test is not a plain gtest you should just build it as a normal executable without registering as a gtest through the catkin_add_gtest macro.