Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

This is probably not an error with the controllers, the error you see is what happens after gazebo segfaults and exists before controllers can be taken down properly. Can you include more of the console outputs?

Does it work with gdb as posted in this question?

On another note, I have seen similar errors when running with nvidia 295.* drivers, the segfault goes away after I downgraded to 285.* drivers. Be careful when downgrading, from experience, 295.* drivers left bits of kernel modules and libraries in miscellaneous places on my system that ultimately confused my system for a while, make sure to uninstall

sudo apt-get remove --purge nvidia-current-updates* nvidia-current*

if you are installing the 285.* drivers manually.

This is probably not an error with the controllers, the error you see is what happens after gazebo segfaults and exists before controllers can be taken down properly. Can you include more of the console outputs?

Does it work with gdb as posted in this question?

On another note, I have seen similar errors when running with nvidia 295.* drivers, the segfault goes away after I downgraded to 285.* drivers. Be careful when downgrading, from experience, 295.* drivers left bits of kernel modules and libraries in miscellaneous places on my system that ultimately confused my system for a while, make sure to uninstalluninstall debs:

sudo apt-get remove --purge nvidia-current-updates* nvidia-current*

if you are installing the 285.* drivers manually.

This is probably not an error with the controllers, the error you see is what happens after gazebo segfaults and exists before controllers can be taken down properly. Can you include more of the console outputs?

Does it work with gdb as posted in this question?

On another note, I have seen similar errors when running with nvidia 295.* drivers, the segfault goes away after I downgraded to 285.* drivers. Be careful when downgrading, from experience, 295.* drivers left bits of kernel modules and libraries in miscellaneous places on my system that ultimately confused my system for a while, make sure to uninstall debs:

sudo apt-get remove --purge nvidia-current-updates* nvidia-current*

if you are installing the 285.* drivers manually.

more info on the bug: this is the backtrace I got when I tried 295.20 driver on my oneiric system:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f51f91c7700 (LWP 30795)]
0x00007f5239dc7c0f in _nv022tls () from /usr/lib/tls/libnvidia-tls.so.295.20
(gdb) bt
#0  0x00007f5239dc7c0f in _nv022tls () from /usr/lib/tls/libnvidia-tls.so.295.20
#1  0x00007f523ceb4a01 in ?? () from /usr/lib/libGL.so.1
#2  0x00007f52475ab5a5 in __libc_fork () at ../nptl/sysdeps/unix/sysv/linux/x86_64/../fork.c:95
#3  0x00007f52475687ab in _IO_new_proc_open (fp=0x7f522c307b20, command=0x7f522c4a0478 "ret=\"/opt/ros/electric/stacks/arm_navigation/joint_normalization_filters/default_plugins.xml\" && echo $ret", mode=<optimized out>)
    at iopopen.c:191
#4  0x00007f5247568b01 in _IO_new_popen (command=0x7f522c4a0478 "ret=\"/opt/ros/electric/stacks/arm_navigation/joint_normalization_filters/default_plugins.xml\" && echo $ret", mode=0x7f52463ccb51 "r") at iopopen.c:312
#5  0x00007f52463b85df in rospack::Package::direct_flags (this=0x7f522c07f9b0, lang=..., attrib=...) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/tools/rospack/rospack.cpp:584
#6  0x00007f52463bf800 in rospack::Package::plugins (this=0x7f522c441d10) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/tools/rospack/rospack.cpp:304
#7  0x00007f52463c0062 in rospack::ROSPack::cmd_plugins (this=0x7f51f91c57e0) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/tools/rospack/rospack.cpp:1164
#8  0x00007f52463c2fd7 in rospack::ROSPack::run (this=0x7f51f91c57e0, argc=4, argv=0x7f522c513200) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/tools/rospack/rospack.cpp:1438
#9  0x00007f52463c3826 in rospack::ROSPack::run (this=0x7f51f91c57e0, cmd=<optimized out>) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/tools/rospack/rospack.cpp:1206
#10 0x00007f5248e43f03 in ros::package::command (_cmd=...) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/core/roslib/src/package.cpp:55
#11 0x00007f5248e44245 in ros::package::command (cmd=<optimized out>, output=...) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/core/roslib/src/package.cpp:73
#12 0x00007f5248e4478b in ros::package::getPlugins (package=..., attribute=<optimized out>, plugins=...) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/core/roslib/src/package.cpp:125
#13 0x00007f5248e4537c in ros::package::getPlugins (package=<optimized out>, attribute=<optimized out>, plugins=...) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/core/roslib/src/package.cpp:113
#14 0x00007f51f6733cae in pluginlib::ClassLoader<filters::FilterBase<double> >::ClassLoader (this=0x7f522c485828, package=..., base_class=..., attrib_name=...)
    at /opt/ros/electric/stacks/pluginlib/include/pluginlib/class_loader_imp.h:51
#15 0x00007f51f6726f0d in FilterChain (data_type=..., this=0x7f522c485828) at /opt/ros/electric/stacks/filters/include/filters/filter_chain.h:53
#16 controller::LaserScannerTrajController::LaserScannerTrajController (this=0x7f522c485670)
    at /tmp/buildd/ros-electric-pr2-controllers-1.6.2/debian/ros-electric-pr2-controllers/opt/ros/electric/stacks/pr2_controllers/pr2_mechanism_controllers/src/laser_scanner_traj_controller.cpp:47
#17 0x00007f51f67274e8 in controller::LaserScannerTrajControllerNode::LaserScannerTrajControllerNode (this=0x7f522c485560)
    at /tmp/buildd/ros-electric-pr2-controllers-1.6.2/debian/ros-electric-pr2-controllers/opt/ros/electric/stacks/pr2_controllers/pr2_mechanism_controllers/src/laser_scanner_traj_controller.cpp:399
#18 0x00007f51f6734e3a in Poco::MetaObject<controller::LaserScannerTrajControllerNode, pr2_controller_interface::Controller>::create (this=<optimized out>) at /opt/ros/electric/stacks/pluginlib/include/Poco/MetaObject.h:165
#19 0x00007f52057abc57 in create (className=..., this=0x7f522431b9b8) at /opt/ros/electric/stacks/pluginlib/include/Poco/ClassLoader.h:360
#20 pluginlib::ClassLoader<pr2_controller_interface::Controller>::createClassInstance (this=0x7f522431b950, lookup_name=..., auto_load=<optimized out>) at /opt/ros/electric/stacks/pluginlib/include/pluginlib/class_loader_imp.h:277
#21 0x00007f520578d68f in pr2_controller_manager::ControllerManager::loadController (this=<optimized out>, name=...)
    at /tmp/buildd/ros-electric-pr2-mechanism-1.5.4/debian/ros-electric-pr2-mechanism/opt/ros/electric/stacks/pr2_mechanism/pr2_controller_manager/src/controller_manager.cpp:289
#22 0x00007f520578ee02 in pr2_controller_manager::ControllerManager::loadControllerSrv (this=0x7f522e917090, req=..., resp=...)
    at /tmp/buildd/ros-electric-pr2-mechanism-1.5.4/debian/ros-electric-pr2-mechanism/opt/ros/electric/stacks/pr2_mechanism/pr2_controller_manager/src/controller_manager.cpp:799
#23 0x00007f52057b4f8b in operator() (a1=<optimized out>, a0=<optimized out>, this=0x7f522430dd98) at /usr/include/boost/function/function_template.hpp:1013
#24 call (params=..., cb=...) at /opt/ros/electric/stacks/ros_comm/clients/cpp/roscpp/include/ros/service_callback_helper.h:142
#25 ros::ServiceCallbackHelperT<ros::ServiceSpec<pr2_mechanism_msgs::LoadControllerRequest_<std::allocator<void> >, pr2_mechanism_msgs::LoadControllerResponse_<std::allocator<void> > > >::call (this=0x7f522430dd90, params=...)
    at /opt/ros/electric/stacks/ros_comm/clients/cpp/roscpp/include/ros/service_callback_helper.h:200
#26 0x00007f5249bbd232 in ros::ServiceCallback::call (this=0x7f522c53aca0)
    at /tmp/buildd/ros-electric-ros-comm-1.6.6/debian/ros-electric-ros-comm/opt/ros/electric/stacks/ros_comm/clients/cpp/roscpp/src/libros/service_publication.cpp:123
#27 0x00007f5249bccc59 in ros::CallbackQueue::callOneCB (this=0x2355b20, tls=0x62f2120)
    at /tmp/buildd/ros-electric-ros-comm-1.6.6/debian/ros-electric-ros-comm/opt/ros/electric/stacks/ros_comm/clients/cpp/roscpp/src/libros/callback_queue.cpp:381
#28 0x00007f5249bce74b in ros::CallbackQueue::callAvailable (this=0x2355b20, timeout=<optimized out>)
    at /tmp/buildd/ros-electric-ros-comm-1.6.6/debian/ros-electric-ros-comm/opt/ros/electric/stacks/ros_comm/clients/cpp/roscpp/src/libros/callback_queue.cpp:333
#29 0x00007f5249b6b42a in ros::spinOnce () at /tmp/buildd/ros-electric-ros-comm-1.6.6/debian/ros-electric-ros-comm/opt/ros/electric/stacks/ros_comm/clients/cpp/roscpp/src/libros/init.cpp:518
#30 0x00007f52059e72bf in gazebo::GazeboRosControllerManager::ControllerManagerROSThread (this=0x7f522e9158b0)
    at /tmp/buildd/ros-electric-pr2-simulator-1.6.0/debian/ros-electric-pr2-simulator/opt/ros/electric/stacks/pr2_simulator/pr2_gazebo_plugins/src/gazebo_ros_controller_manager.cpp:509
#31 0x00007f5249280ba9 in thread_proxy () from /usr/lib/libboost_thread.so.1.46.1
#32 0x00007f524a29aefc in start_thread (arg=0x7f51f91c7700) at pthread_create.c:304
#33 0x00007f52475df89d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#34 0x0000000000000000 in ?? ()
(gdb)

and here is the discussion thread I found with a small glut test to reproduce the problem.

This is probably not an error with the controllers, the error you see is what happens after gazebo segfaults and exists exits before controllers can be taken down properly. Can you include more of the console outputs?

Does it work with gdb as posted in this question?

On another note, I have seen similar errors when running with nvidia 295.* drivers, the segfault goes away after I downgraded to 285.* drivers. Be careful when downgrading, from experience, 295.* drivers left bits of kernel modules and libraries in miscellaneous places on my system that ultimately confused my system for a while, make sure to uninstall debs:

sudo apt-get remove --purge nvidia-current-updates* nvidia-current*

if you are installing the 285.* drivers manually.

more info on the bug: this is the backtrace I got when I tried 295.20 driver on my oneiric system:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f51f91c7700 (LWP 30795)]
0x00007f5239dc7c0f in _nv022tls () from /usr/lib/tls/libnvidia-tls.so.295.20
(gdb) bt
#0  0x00007f5239dc7c0f in _nv022tls () from /usr/lib/tls/libnvidia-tls.so.295.20
#1  0x00007f523ceb4a01 in ?? () from /usr/lib/libGL.so.1
#2  0x00007f52475ab5a5 in __libc_fork () at ../nptl/sysdeps/unix/sysv/linux/x86_64/../fork.c:95
#3  0x00007f52475687ab in _IO_new_proc_open (fp=0x7f522c307b20, command=0x7f522c4a0478 "ret=\"/opt/ros/electric/stacks/arm_navigation/joint_normalization_filters/default_plugins.xml\" && echo $ret", mode=<optimized out>)
    at iopopen.c:191
#4  0x00007f5247568b01 in _IO_new_popen (command=0x7f522c4a0478 "ret=\"/opt/ros/electric/stacks/arm_navigation/joint_normalization_filters/default_plugins.xml\" && echo $ret", mode=0x7f52463ccb51 "r") at iopopen.c:312
#5  0x00007f52463b85df in rospack::Package::direct_flags (this=0x7f522c07f9b0, lang=..., attrib=...) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/tools/rospack/rospack.cpp:584
#6  0x00007f52463bf800 in rospack::Package::plugins (this=0x7f522c441d10) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/tools/rospack/rospack.cpp:304
#7  0x00007f52463c0062 in rospack::ROSPack::cmd_plugins (this=0x7f51f91c57e0) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/tools/rospack/rospack.cpp:1164
#8  0x00007f52463c2fd7 in rospack::ROSPack::run (this=0x7f51f91c57e0, argc=4, argv=0x7f522c513200) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/tools/rospack/rospack.cpp:1438
#9  0x00007f52463c3826 in rospack::ROSPack::run (this=0x7f51f91c57e0, cmd=<optimized out>) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/tools/rospack/rospack.cpp:1206
#10 0x00007f5248e43f03 in ros::package::command (_cmd=...) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/core/roslib/src/package.cpp:55
#11 0x00007f5248e44245 in ros::package::command (cmd=<optimized out>, output=...) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/core/roslib/src/package.cpp:73
#12 0x00007f5248e4478b in ros::package::getPlugins (package=..., attribute=<optimized out>, plugins=...) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/core/roslib/src/package.cpp:125
#13 0x00007f5248e4537c in ros::package::getPlugins (package=<optimized out>, attribute=<optimized out>, plugins=...) at /tmp/buildd/ros-electric-ros-1.6.8/debian/ros-electric-ros/opt/ros/electric/ros/core/roslib/src/package.cpp:113
#14 0x00007f51f6733cae in pluginlib::ClassLoader<filters::FilterBase<double> >::ClassLoader (this=0x7f522c485828, package=..., base_class=..., attrib_name=...)
    at /opt/ros/electric/stacks/pluginlib/include/pluginlib/class_loader_imp.h:51
#15 0x00007f51f6726f0d in FilterChain (data_type=..., this=0x7f522c485828) at /opt/ros/electric/stacks/filters/include/filters/filter_chain.h:53
#16 controller::LaserScannerTrajController::LaserScannerTrajController (this=0x7f522c485670)
    at /tmp/buildd/ros-electric-pr2-controllers-1.6.2/debian/ros-electric-pr2-controllers/opt/ros/electric/stacks/pr2_controllers/pr2_mechanism_controllers/src/laser_scanner_traj_controller.cpp:47
#17 0x00007f51f67274e8 in controller::LaserScannerTrajControllerNode::LaserScannerTrajControllerNode (this=0x7f522c485560)
    at /tmp/buildd/ros-electric-pr2-controllers-1.6.2/debian/ros-electric-pr2-controllers/opt/ros/electric/stacks/pr2_controllers/pr2_mechanism_controllers/src/laser_scanner_traj_controller.cpp:399
#18 0x00007f51f6734e3a in Poco::MetaObject<controller::LaserScannerTrajControllerNode, pr2_controller_interface::Controller>::create (this=<optimized out>) at /opt/ros/electric/stacks/pluginlib/include/Poco/MetaObject.h:165
#19 0x00007f52057abc57 in create (className=..., this=0x7f522431b9b8) at /opt/ros/electric/stacks/pluginlib/include/Poco/ClassLoader.h:360
#20 pluginlib::ClassLoader<pr2_controller_interface::Controller>::createClassInstance (this=0x7f522431b950, lookup_name=..., auto_load=<optimized out>) at /opt/ros/electric/stacks/pluginlib/include/pluginlib/class_loader_imp.h:277
#21 0x00007f520578d68f in pr2_controller_manager::ControllerManager::loadController (this=<optimized out>, name=...)
    at /tmp/buildd/ros-electric-pr2-mechanism-1.5.4/debian/ros-electric-pr2-mechanism/opt/ros/electric/stacks/pr2_mechanism/pr2_controller_manager/src/controller_manager.cpp:289
#22 0x00007f520578ee02 in pr2_controller_manager::ControllerManager::loadControllerSrv (this=0x7f522e917090, req=..., resp=...)
    at /tmp/buildd/ros-electric-pr2-mechanism-1.5.4/debian/ros-electric-pr2-mechanism/opt/ros/electric/stacks/pr2_mechanism/pr2_controller_manager/src/controller_manager.cpp:799
#23 0x00007f52057b4f8b in operator() (a1=<optimized out>, a0=<optimized out>, this=0x7f522430dd98) at /usr/include/boost/function/function_template.hpp:1013
#24 call (params=..., cb=...) at /opt/ros/electric/stacks/ros_comm/clients/cpp/roscpp/include/ros/service_callback_helper.h:142
#25 ros::ServiceCallbackHelperT<ros::ServiceSpec<pr2_mechanism_msgs::LoadControllerRequest_<std::allocator<void> >, pr2_mechanism_msgs::LoadControllerResponse_<std::allocator<void> > > >::call (this=0x7f522430dd90, params=...)
    at /opt/ros/electric/stacks/ros_comm/clients/cpp/roscpp/include/ros/service_callback_helper.h:200
#26 0x00007f5249bbd232 in ros::ServiceCallback::call (this=0x7f522c53aca0)
    at /tmp/buildd/ros-electric-ros-comm-1.6.6/debian/ros-electric-ros-comm/opt/ros/electric/stacks/ros_comm/clients/cpp/roscpp/src/libros/service_publication.cpp:123
#27 0x00007f5249bccc59 in ros::CallbackQueue::callOneCB (this=0x2355b20, tls=0x62f2120)
    at /tmp/buildd/ros-electric-ros-comm-1.6.6/debian/ros-electric-ros-comm/opt/ros/electric/stacks/ros_comm/clients/cpp/roscpp/src/libros/callback_queue.cpp:381
#28 0x00007f5249bce74b in ros::CallbackQueue::callAvailable (this=0x2355b20, timeout=<optimized out>)
    at /tmp/buildd/ros-electric-ros-comm-1.6.6/debian/ros-electric-ros-comm/opt/ros/electric/stacks/ros_comm/clients/cpp/roscpp/src/libros/callback_queue.cpp:333
#29 0x00007f5249b6b42a in ros::spinOnce () at /tmp/buildd/ros-electric-ros-comm-1.6.6/debian/ros-electric-ros-comm/opt/ros/electric/stacks/ros_comm/clients/cpp/roscpp/src/libros/init.cpp:518
#30 0x00007f52059e72bf in gazebo::GazeboRosControllerManager::ControllerManagerROSThread (this=0x7f522e9158b0)
    at /tmp/buildd/ros-electric-pr2-simulator-1.6.0/debian/ros-electric-pr2-simulator/opt/ros/electric/stacks/pr2_simulator/pr2_gazebo_plugins/src/gazebo_ros_controller_manager.cpp:509
#31 0x00007f5249280ba9 in thread_proxy () from /usr/lib/libboost_thread.so.1.46.1
#32 0x00007f524a29aefc in start_thread (arg=0x7f51f91c7700) at pthread_create.c:304
#33 0x00007f52475df89d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#34 0x0000000000000000 in ?? ()
(gdb)

and here is the discussion thread I found with a small glut test to reproduce the problem.