Robotics StackExchange | Archived questions

turtlebot minimal.launch race condition

Has anyone else encountered this race condition when running roslaunch turtlebot_bringup minimal.launch?

The problem is that the turtlebot's interactions manager is being started, failing to connect to the app manager and committing suicide all before the app manager is able to start.

On line 79 of /opt/ros/indigo/lib/python2.7/dist-packages/roconinteractions/rapphandler.py it tries to wait but fails because the app manager hasn't yet been able to open its network ports.

start_rapp_service_name = rocon_python_comms.find_service('rocon_app_manager_msgs/StartRapp', timeout=rospy.rostime.Duration(60.0), unique=True)

???@???:~$ roslaunch turtlebot_bringup minimal.launch --screen
... logging to /home/???/.ros/log/d342f098-400c-11e5-99f4-e006e68921e1/roslaunch-???-18545.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://???:51355/

SUMMARY
========

PARAMETERS
 * /app_manager/auto_rapp_installation: False
 * /app_manager/auto_start_rapp: 
 * /app_manager/capability_server_name: capability_server
 * /app_manager/local_remote_controllers_only: False
 * /app_manager/preferred: [{'rocon_apps/chi...
 * /app_manager/rapp_package_blacklist: []
 * /app_manager/rapp_package_whitelist: ['rocon_apps', 't...
 * /app_manager/robot_icon: turtlebot_bringup...
 * /app_manager/robot_name: michelangelo
 * /app_manager/robot_type: turtlebot
 * /app_manager/screen: True
 * /app_manager/simulation: False
 * /app_manager/use_gateway_uuids: False
 * /capability_server/blacklist: ['std_capabilitie...
 * /capability_server/defaults/kobuki_capabilities/KobukiBringup: kobuki_capabiliti...
 * /capability_server/defaults/kobuki_capabilities/KobukiBumper: kobuki_capabiliti...
 * /capability_server/defaults/kobuki_capabilities/KobukiCliffDetection: kobuki_capabiliti...
 * /capability_server/defaults/kobuki_capabilities/KobukiLED1: kobuki_capabiliti...
 * /capability_server/defaults/kobuki_capabilities/KobukiLED2: kobuki_capabiliti...
 * /capability_server/defaults/kobuki_capabilities/KobukiLED: kobuki_capabiliti...
 * /capability_server/defaults/kobuki_capabilities/KobukiWheelDropDetection: kobuki_capabiliti...
 * /capability_server/defaults/std_capabilities/Diagnostics: turtlebot_capabil...
 * /capability_server/defaults/std_capabilities/DifferentialMobileBase: kobuki_capabiliti...
 * /capability_server/defaults/std_capabilities/LaserSensor: turtlebot_capabil...
 * /capability_server/defaults/std_capabilities/RGBDSensor: turtlebot_capabil...
 * /capability_server/defaults/std_capabilities/RobotStatePublisher: turtlebot_capabil...
 * /capability_server/defaults/turtlebot_capabilities/TurtleBotBringup: turtlebot_capabil...
 * /capability_server/nodelet_manager_name: capability_server...
 * /capability_server/package_whitelist: ['kobuki_capabili...
 * /cmd_vel_mux/yaml_cfg_file: /opt/ros/indigo/s...
 * /description: Kick-ass ROS turtle
 * /diagnostic_aggregator/analyzers/digital_io/path: Digital IO
 * /diagnostic_aggregator/analyzers/digital_io/startswith: ['Digital Outputs']
 * /diagnostic_aggregator/analyzers/digital_io/timeout: 5.0
 * /diagnostic_aggregator/analyzers/digital_io/type: diagnostic_aggreg...
 * /diagnostic_aggregator/analyzers/mode/path: Mode
 * /diagnostic_aggregator/analyzers/mode/startswith: ['Operating Mode']
 * /diagnostic_aggregator/analyzers/mode/timeout: 5.0
 * /diagnostic_aggregator/analyzers/mode/type: diagnostic_aggreg...
 * /diagnostic_aggregator/analyzers/nodes/contains: ['Node']
 * /diagnostic_aggregator/analyzers/nodes/path: Nodes
 * /diagnostic_aggregator/analyzers/nodes/timeout: 5.0
 * /diagnostic_aggregator/analyzers/nodes/type: diagnostic_aggreg...
 * /diagnostic_aggregator/analyzers/power/path: Power System
 * /diagnostic_aggregator/analyzers/power/startswith: ['Battery', 'Char...
 * /diagnostic_aggregator/analyzers/power/timeout: 5.0
 * /diagnostic_aggregator/analyzers/power/type: diagnostic_aggreg...
 * /diagnostic_aggregator/analyzers/sensors/path: Sensors
 * /diagnostic_aggregator/analyzers/sensors/startswith: ['Cliff Sensor', ...
 * /diagnostic_aggregator/analyzers/sensors/timeout: 5.0
 * /diagnostic_aggregator/analyzers/sensors/type: GenericAnalyzer
 * /diagnostic_aggregator/base_path: 
 * /diagnostic_aggregator/pub_rate: 1.0
 * /icon: turtlebot_bringup...
 * /interactions/interactions: ['turtlebot_bring...
 * /interactions/pairing: True
 * /interactions/rosbridge_address: localhost
 * /interactions/rosbridge_port: 9090
 * /interactions/webserver_address: webapp.robotconce...
 * /name: michelangelo
 * /robot/name: turtlebot
 * /robot/type: turtlebot
 * /robot_description: <?xml version="1....
 * /robot_pose_ekf/freq: 10.0
 * /robot_pose_ekf/imu_used: True
 * /robot_pose_ekf/odom_used: True
 * /robot_pose_ekf/output_frame: odom
 * /robot_pose_ekf/publish_tf: True
 * /robot_pose_ekf/sensor_timeout: 1.0
 * /robot_pose_ekf/vo_used: False
 * /robot_state_publisher/publish_frequency: 5.0
 * /rosdistro: indigo
 * /rosversion: 1.11.13
 * /turtlebot_laptop_battery/acpi_path: /proc/acpi/batter...
 * /turtlebot_node/bonus: False
 * /turtlebot_node/port: /dev/ttyUSB0
 * /turtlebot_node/update_rate: 30.0
 * /use_sim_time: False
 * /zeroconf/zeroconf/services: [{'domain': 'loca...

NODES
  /zeroconf/
    zeroconf (zeroconf_avahi/zeroconf)
  /
    app_manager (rocon_app_manager/rapp_manager.py)
    capability_server (capabilities/capability_server)
    cmd_vel_mux (nodelet/nodelet)
    create_load_calibration (create_node/load_calib.py)
    diagnostic_aggregator (diagnostic_aggregator/aggregator_node)
    interactions (rocon_interactions/interactions_manager.py)
    kinect_breaker_enabler (create_node/kinect_breaker_enabler.py)
    master (rocon_master_info/master.py)
    mobile_base_nodelet_manager (nodelet/nodelet)
    robot_pose_ekf (robot_pose_ekf/robot_pose_ekf)
    robot_state_publisher (robot_state_publisher/robot_state_publisher)
    turtlebot_laptop_battery (laptop_battery_monitor/laptop_battery.py)
    turtlebot_node (create_node/turtlebot_node.py)

auto-starting new master
process[master]: started with pid [18560]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to d342f098-400c-11e5-99f4-e006e68921e1
process[rosout-1]: started with pid [18573]
started core service [/rosout]
process[robot_state_publisher-2]: started with pid [18591]
/opt/ros/indigo/lib/robot_state_publisher/robot_state_publisher
process[diagnostic_aggregator-3]: started with pid [18617]
process[turtlebot_node-4]: started with pid [18632]
process[kinect_breaker_enabler-5]: started with pid [18637]
[ WARN] [1439285962.243972046]: Analyzer specification should now include the package name. You are using a deprecated API. Please switch from GenericAnalyzer to diagnostic_aggregator/GenericAnalyzer in your Analyzer specification.
process[create_load_calibration-6]: started with pid [18709]
process[robot_pose_ekf-7]: started with pid [18710]
[ INFO] [1439285963.303904517]: output frame: odom
[ INFO] [1439285963.304997865]: base frame: base_footprint
process[mobile_base_nodelet_manager-8]: started with pid [18773]
[ INFO] [1439285964.163937887]: Initializing nodelet with 4 worker threads.
process[cmd_vel_mux-9]: started with pid [18794]
[ INFO] [1439285964.524452094]: Loading nodelet /cmd_vel_mux of type yocs_cmd_vel_mux/CmdVelMuxNodelet to manager mobile_base_nodelet_manager with the following remappings:
[ INFO] [1439285964.524987382]: /cmd_vel_mux/output -> /mobile_base/commands/velocity
process[turtlebot_laptop_battery-10]: started with pid [18811]
process[capability_server-11]: started with pid [18849]
[INFO] [WallTime: 1439285965.544061] serial port: /dev/ttyUSB0
[INFO] [WallTime: 1439285965.545469] update_rate: 30.0
[INFO] [WallTime: 1439285965.547118] drive mode: twist
[INFO] [WallTime: 1439285965.548837] has gyro: True
[ INFO] [1439285965.725585855]: CmdVelMux : (re)configured [/opt/ros/indigo/share/turtlebot_bringup/param/mux.yaml]
[WARN] [WallTime: 1439285965.880501] Create : robot not connected yet, sci not available
[INFO] [WallTime: 1439285965.948326] self.gyro_measurement_range 150.000000
[INFO] [WallTime: 1439285965.951274] self.gyro_scale_correction 1.350000
process[app_manager-12]: started with pid [18945]
process[master-13]: started with pid [18982]
[INFO] [WallTime: 1439285966.624085] self.gyro_measurement_range 250.000000
[INFO] [WallTime: 1439285966.626552] self.gyro_scale_correction 2.500000
[INFO] [WallTime: 1439285966.758499] Turtlebot calibration successfully loaded from /home/mmcgill/.ros/turtlebot_create/A00364908685114A.yaml
process[interactions-14]: started with pid [19020]
process[zeroconf/zeroconf-15]: started with pid [19027]
[create_load_calibration-6] process has finished cleanly
log file: /home/mmcgill/.ros/log/d342f098-400c-11e5-99f4-e006e68921e1/create_load_calibration-6*.log
[ INFO] [1439285968.802636787]: Zeroconf: service successfully established [michelangelo][_ros-master._tcp][11311]
[INFO] [WallTime: 1439285968.860695] Rapp Manager : indexing rapps...
[WARN] [WallTime: 1439285968.898479] Create : robot not connected yet, sci not available
... logging to /home/mmcgill/.ros/log/d342f098-400c-11e5-99f4-e006e68921e1/roslaunch-michelangelo-19059.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

[INFO] [WallTime: 1439285971.915880] Setting turtlebot to full mode.
started roslaunch server http://michelangelo:57274/

SUMMARY
========

PARAMETERS
 * /rosdistro: indigo
 * /rosversion: 1.11.13

NODES
  /
    capability_server_nodelet_manager (nodelet/nodelet)

ROS_MASTER_URI=http://localhost:11311

core service [/rosout] found
[ INFO] [1439285972.508794107]: Initializing Odom sensor
[ INFO] [1439285972.509065232]: Odom sensor activated
[ INFO] [1439285972.516379283]: Initializing Imu sensor
[ INFO] [1439285972.516772540]: Imu sensor activated
[ INFO] [1439285972.607310887]: Kalman filter initialized with odom measurement
[ERROR] [WallTime: 1439285973.066136] Interactions : timed out trying to find the rapp manager start_rapp, stop_rapp services and status topic [unable to communicate with the master [no provider]]
Traceback (most recent call last):
  File "/opt/ros/indigo/lib/rocon_interactions/interactions_manager.py", line 20, in <module>
    interactions_manager = rocon_interactions.InteractionsManager()
  File "/opt/ros/indigo/lib/python2.7/dist-packages/rocon_interactions/manager.py", line 75, in __init__
    self._rapp_handler = RappHandler(self._rapp_manager_status_update_callback)
  File "/opt/ros/indigo/lib/python2.7/dist-packages/rocon_interactions/rapp_handler.py", line 75, in __init__
    thread = threading.Thread(target=self._setup_rapp_manager_connections())
  File "/opt/ros/indigo/lib/python2.7/dist-packages/rocon_interactions/rapp_handler.py", line 87, in _setup_rapp_manager_connections
    self.start_rapp = rospy.ServiceProxy(start_rapp_service_name, rocon_app_manager_srvs.StartRapp)
UnboundLocalError: local variable 'start_rapp_service_name' referenced before assignment
[kinect_breaker_enabler-5] process has finished cleanly
log file: /home/mmcgill/.ros/log/d342f098-400c-11e5-99f4-e006e68921e1/kinect_breaker_enabler-5*.log
process[capability_server_nodelet_manager-1]: started with pid [19595]
[interactions-14] process has died [pid 19020, exit code 1, cmd /opt/ros/indigo/lib/rocon_interactions/interactions_manager.py __name:=interactions __log:=/home/mmcgill/.ros/log/d342f098-400c-11e5-99f4-e006e68921e1/interactions-14.log].
log file: /home/mmcgill/.ros/log/d342f098-400c-11e5-99f4-e006e68921e1/interactions-14*.log
[ INFO] [1439285974.231480172]: Initializing nodelet with 4 worker threads.
[INFO] [WallTime: 1439285975.027523] Rapp Manager : determining runnable rapps...
[INFO] [WallTime: 1439285975.289974] Spec 'std_capabilities/MultiEchoLaserSensor' is in the blacklist, skipping.
[INFO] [WallTime: 1439285975.291981] Spec 'std_capabilities/Navigation2D' is in the blacklist, skipping.
[INFO] [WallTime: 1439285975.377872] For each available interface, the default provider:
[INFO] [WallTime: 1439285975.379781] 'kobuki_capabilities/KobukiLED2'
[INFO] [WallTime: 1439285975.381893]   => 'kobuki_capabilities/kobuki_led2'
[INFO] [WallTime: 1439285975.383619] 
[INFO] [WallTime: 1439285975.385236] 'kobuki_capabilities/KobukiBringup'
[INFO] [WallTime: 1439285975.387322]   => 'kobuki_capabilities/kobuki_bringup'
[INFO] [WallTime: 1439285975.389007] 
[INFO] [WallTime: 1439285975.391075] 'kobuki_capabilities/KobukiLED1'
[INFO] [WallTime: 1439285975.392757]   => 'kobuki_capabilities/kobuki_led1'
[INFO] [WallTime: 1439285975.394899] 
[INFO] [WallTime: 1439285975.396562] 'kobuki_capabilities/KobukiLED'
[INFO] [WallTime: 1439285975.398456]   => 'kobuki_capabilities/kobuki_led'
[INFO] [WallTime: 1439285975.400151] 
[INFO] [WallTime: 1439285975.402090] 'kobuki_capabilities/KobukiBumper'
[INFO] [WallTime: 1439285975.403960]   => 'kobuki_capabilities/kobuki_bumper'
[INFO] [WallTime: 1439285975.405827] 
[INFO] [WallTime: 1439285975.407387] 'std_capabilities/RGBDSensor'
[INFO] [WallTime: 1439285975.408870]   => 'turtlebot_capabilities/rgbd_sensor'
[INFO] [WallTime: 1439285975.410807] 
[INFO] [WallTime: 1439285975.413072] 'std_capabilities/LaserSensor'
[INFO] [WallTime: 1439285975.414950]   => 'turtlebot_capabilities/depthimage_to_laserscan'
[INFO] [WallTime: 1439285975.416560] 
[INFO] [WallTime: 1439285975.418307] 'kobuki_capabilities/KobukiWheelDropDetection'
[INFO] [WallTime: 1439285975.420212]   => 'kobuki_capabilities/kobuki_wheel_drop_detection'
[INFO] [WallTime: 1439285975.422044] 
[INFO] [WallTime: 1439285975.424090] 'std_capabilities/RobotStatePublisher'
[INFO] [WallTime: 1439285975.426986]   => 'turtlebot_capabilities/robot_state_publisher'
[INFO] [WallTime: 1439285975.429069] 
[INFO] [WallTime: 1439285975.430988] 'std_capabilities/DifferentialMobileBase'
[INFO] [WallTime: 1439285975.432924]   => 'kobuki_capabilities/kobuki_differential_mobile_base'
[INFO] [WallTime: 1439285975.434831] 
[INFO] [WallTime: 1439285975.436587] 'kobuki_capabilities/KobukiCliffDetection'
[INFO] [WallTime: 1439285975.438373]   => 'kobuki_capabilities/kobuki_cliff_detection'
[INFO] [WallTime: 1439285975.439838] 
[INFO] [WallTime: 1439285975.442056] 'std_capabilities/Diagnostics'
[INFO] [WallTime: 1439285975.443978]   => 'turtlebot_capabilities/diagnostics'
[INFO] [WallTime: 1439285975.446115] 
[INFO] [WallTime: 1439285975.448069] 'turtlebot_capabilities/TurtleBotBringup'
[INFO] [WallTime: 1439285975.450008]   => 'turtlebot_capabilities/turtlebot2_bringup'
[INFO] [WallTime: 1439285975.453540] 
[INFO] [WallTime: 1439285975.564353] Capability Server Ready
[INFO] [WallTime: 1439285975.808296] Request to establish a bond
/opt/ros/indigo/lib/python2.7/dist-packages/bondpy/bondpy.py:114: SyntaxWarning: The publisher should be created with an explicit keyword argument 'queue_size'. Please see http://wiki.ros.org/rospy/Overview/Publishers%20and%20Subscribers for more information.
  self.pub = rospy.Publisher(self.topic, Status)
/opt/ros/indigo/lib/python2.7/dist-packages/bondpy/bondpy.py:114: SyntaxWarning: The publisher should be created with an explicit keyword argument 'queue_size'. Please see http://wiki.ros.org/rospy/Overview/Publishers%20and%20Subscribers for more information.
  self.pub = rospy.Publisher(self.topic, Status)
[INFO] [WallTime: 1439285976.867660] Bond formed with bond_id of 'e0b26a6a-400c-11e5-9851-e006e68921e1'
[INFO] [WallTime: 1439285976.917467] Servicing request for capability specs...
[INFO] [WallTime: 1439285977.701579] Rapp Manager : 'turtlebot_rapps/xbox360_teleop' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.703612] Rapp Manager : 'turtlebot_rapps/follower' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.705426] Rapp Manager : 'turtlebot_rapps/map_nav' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.707333] Rapp Manager : 'rocon_apps/meow_chirp' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.709610] Rapp Manager : 'turtlebot_rapps/3dsensor' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.711972] Rapp Manager : 'turtlebot_rapps/ps3_teleop' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.713968] Rapp Manager : 'turtlebot_rapps/make_a_map' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.715479] Rapp Manager : 'turtlebot_rapps/video_teleop' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.717025] Rapp Manager : 'rocon_apps/moo_chirp' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.718864] Rapp Manager : 'rocon_apps/angry_cat_chirp' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.720262] Rapp Manager : 'rocon_apps/lion_chirp' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.722179] Rapp Manager : 'turtlebot_rapps/panorama' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.723570] Rapp Manager : 'rocon_apps/listener' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.725384] Rapp Manager : 'turtlebot_rapps/auto_docking' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.727407] Rapp Manager : 'rocon_apps/talker' added to the list of runnable apps.
[INFO] [WallTime: 1439285977.729303] Rapp Manager: 'rocon_apps/chirp' -> 'rocon_apps/moo_chirp'
[INFO] [WallTime: 1439285977.842040] Rapp Manager : initialised.
^A^C[zeroconf/zeroconf-15] killing on exit
[master-13] killing on exit
[app_manager-12] killing on exit
[capability_server-11] killing on exit
[turtlebot_laptop_battery-10] killing on exit
[cmd_vel_mux-9] killing on exit
[mobile_base_nodelet_manager-8] killing on exit
[robot_pose_ekf-7] killing on exit
[turtlebot_node-4] killing on exit
[ INFO] [1439286057.740039067]: Unloading nodelet /cmd_vel_mux from manager mobile_base_nodelet_manager
[diagnostic_aggregator-3] killing on exit
[capability_server_nodelet_manager-1] killing on exit
[robot_state_publisher-2] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done
[rosout-1] killing on exit
[master] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done

This is on a machine running Ubuntu 14.04 and ROS indigo with the latest turtlebot debs.

Asked by Matt on 2015-08-11 05:06:00 UTC

Comments

Answers

It might have racing condition in https://github.com/robotics-in-concert/rocon_tools/blob/develop/rocon_interactions/src/rocon_interactions/rapp_handler.py#L78-L83.

However, it waits 60 seconds for the topics to be ready. which should be enough time for turtlebot to be started in most of recent laptops. So I would suggest to try with other laptops. Or increase the timeout in source level.

Asked by jihoonl on 2015-09-02 18:57:55 UTC

Comments