In general you should right unit tests using gtest directly. You will then just need to register them as unit tests with the buildsystem (rosbuild or catkin)
rosunit is a test runner which we use in the continuous integration process, but it requires gtest compatible unit tests. As a developer you can use it to run your tests, but it's not part of how you consider running your unit tests.
rostest is how you can write tests which run multiple processes and test ROS based communications. Within a rostest you should have a gtest node which is doing asserts, and other modules to fill out the other side of the communications. There's good focused tutorials for rostest on it's homepage.