possible cause for /scan not sending new messages ?

asked 2021-06-07 12:32:12 -0500

fourchette gravatar image

hello,

I'm using a Slamtec A1M8 to get some /scan messages on ROS neotic running on a headless ubuntu 20 server on a raspberry pi running on a power bank. installation of ROS was made using the standard ubuntu procedure => http://wiki.ros.org/noetic/Installati... the A1M8 is connected through USB using the provided hardware adapter.

This is the driver I use => https://github.com/Slamtec/rplidar_ros that publishes sensor_msgs/LaserScan

I have this "device" running somewhere I cannot physically go to due to covid-19 restrictions. It is physically attached to some sort of a trolley that is supposed be manually moved around by a person (there are vibrations). I'm using the A1M8 to detect gestures of that person.

all ros related programs are started with a ros launch file that is started when the raspberry pi is started.

The behavior I have been reported is: as the person starts walking around, randomly after about a few minutes, the detection stops working. then if you power off/on the system, it will work again for a minutes and then detection stops working again

I checked in logs => I do not see any error messages. The only thing I noticed after the behavior has appeared is that a rostopic hz /scan typically answers

no new messages
no new messages
no new messages

instead of something like

average rate: 6.404
        min: 0.138s max: 0.177s std dev: 0.01025s window: 77
average rate: 6.404
        min: 0.138s max: 0.177s std dev: 0.01021s window: 83

Basically the lidar stopped sending data. Visually it is still rotating. I have not been reported any funny sound. It visually looks ok. all cables are correctly connected (nothing seems poorly connected, everything is "clicked" in)

In my hometown where I am located, I am totally unable to reproduce this behavior at all.

If you don't physically move around the trolley this device is attached to => the behavior does not reproduce 😮 !!! I tried exchanging all components (other raspberry pi, rplidar, batteries, cables, everything). I basically sent over another entire box I had personally checked where I never reproduced the behavior even under terrible vibrations/stress => behavior does reproduce once received over there! 💩

I did make a rosbag but i'm unable to make sense of it. (in case you want to have a look => https://drive.google.com/file/d/1UPuC... )

I am stuck. Anyone got any clue ?

Why a node would stop sending messages without any warning in the log? How come I am unable to reproduce the issue in my lab? What would you suggest I check/change?

edit retag flag offensive close merge delete

Comments

Have you tried playing the various topics starting with /scan from that rosbag back as input to your hometown system?

It may be a coincidence but one of your grids appears and disappears at exactly the same time as the lidar stops. Try watching your topics on RViz against:

rosbag play -d 1 -r 0.1 -s 234.5 -u 0.8 2021-06-07-18-31-15.bag

Hence why I recommend you play that rosbag into your bench system.

Another consideration is the issue may be related to: https://github.com/Slamtec/rplidar_ro... I note they report intermittent failures when objects (a particular persons hand as described) comes too close to the Lidar. I can see that you have something quite near at the 4 oclock position and possibly getting some specular artifacts or bad returns which appear as linear rows of points pointing to the Lidar.

Watch on RViz against:

rosbag play ...
(more)
James NT gravatar image James NT  ( 2021-06-08 06:51:46 -0500 )edit

thanks for your feedback. I need to sharpen my skills rosbag skills :D

We nailed the issue to a communication (usb?) issue. If you manually toy/tickle the cable (in particular the connection between the male micro-usb on the cable and its female counterpart on the slamtec adapter board, you can clearly see a usb disconnect/connect separated by ~100ms in /var/log/syslog. if you do not apply the udev rules (https://github.com/Slamtec/rplidar_ro...) then you clearly see the lidar changing from /dev/ttyUSB0 -> /dev/ttyUSB1 -> /dev/ttyUSB0 -> etc. if you apply the udev rules, then I guess the symlink stays on /dev/rplidar but is likely destroyed and recreated by linux.

it's basically the same as a USB unplug/replug, or with white ribbon.

I tried to declare rplidarNode as "respawn="true" respawn_delay="2.0", and then implement a watchdog to kill ...(more)

fourchette gravatar image fourchette  ( 2021-06-13 13:37:20 -0500 )edit

I feel your pain, It's why I tend to field Sick Tim or LMS Lidars for real World applications. They are 5 times the cost but 1/10 the issues. Are you able to simply completely power cycle the Slamtec unit? On the other hand, Micro-USB plugs would seem pretty reliable, I would expect you have another electrical or static issue associated with it given you replaced the unit at one stage.

James NT gravatar image James NT  ( 2021-06-15 10:46:28 -0500 )edit