ROS Answers: Open Source Q&A Forum - RSS feedhttps://answers.ros.org/questions/Open source question and answer forum written in Python and DjangoenROS Answers is licensed under Creative Commons Attribution 3.0Thu, 09 Aug 2018 10:41:06 -0500Getting information out of the /scan topichttps://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/I would like to know the distance to the nearest obstacle in 8 wedges around the robot, i.e. N,E,W,S,NE,NW,SE,SW. My robot is a Turtlebot3 with just a Lidar but this question is general.
What I've done so far to see how far it gets is taking the mean value in each of the wedges and treating that as the distance of the nearest obstacle. So essentially for each scan I get back 8 numbers. I ignore values that are below or above the respective thresholds as well as infinity and NaN.
This is not good enough because of severe noise in the results of the scan (or bugs in my code. For now I am assuming noisy sensors.)
In consecutive scans I may get radically different vectors of 8 values (fake example data following:)
[0.5, 0.4, 0.3, 1.2, 1.3, 1.4, 2.0]
[2.5, 1.4, 0.3, 1.3, 2.3, 1.4, 2.0]
Now remember that each of those numbers is a mean of 360/8 Lidar Readings. So there is some smoothing based on angle already happening. But it looks like I need smoothing over time too. Before I go reinventing the wheel I am looking for advice.
There's a nice laser_filters packages which looks very relevant but I am not sure its what I need.
### Followup
An idea suggested below by @Geoff goes like this:
1. Compute the vector of 8 values at time t. This can be done by for example taking the median +/- std dev or just the average of values in the second and third quartile.
1. Have time based filter by having a simple sliding window average or do something fancier with a Kaman filter.
Thoughts?Wed, 08 Aug 2018 19:37:19 -0500https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/Comment by PeteBlackerThe3rd for <p>I would like to know the distance to the nearest obstacle in 8 wedges around the robot, i.e. N,E,W,S,NE,NW,SE,SW. My robot is a Turtlebot3 with just a Lidar but this question is general.</p>
<p>What I've done so far to see how far it gets is taking the mean value in each of the wedges and treating that as the distance of the nearest obstacle. So essentially for each scan I get back 8 numbers. I ignore values that are below or above the respective thresholds as well as infinity and NaN.</p>
<p>This is not good enough because of severe noise in the results of the scan (or bugs in my code. For now I am assuming noisy sensors.)</p>
<p>In consecutive scans I may get radically different vectors of 8 values (fake example data following:)</p>
<pre><code>[0.5, 0.4, 0.3, 1.2, 1.3, 1.4, 2.0]
[2.5, 1.4, 0.3, 1.3, 2.3, 1.4, 2.0]
</code></pre>
<p>Now remember that each of those numbers is a mean of 360/8 Lidar Readings. So there is some smoothing based on angle already happening. But it looks like I need smoothing over time too. Before I go reinventing the wheel I am looking for advice.</p>
<p>There's a nice laser_filters packages which looks very relevant but I am not sure its what I need.</p>
<h3>Followup</h3>
<p>An idea suggested below by <a href="/users/20134/geoff/">@Geoff</a> goes like this:</p>
<ol>
<li>Compute the vector of 8 values at time t. This can be done by for example taking the median +/- std dev or just the average of values in the second and third quartile.</li>
<li>Have time based filter by having a simple sliding window average or do something fancier with a Kaman filter.</li>
</ol>
<p>Thoughts?</p>
https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/?comment=300157#post-id-300157I've used a lot of lidars and I'd expect an error of a few cm at most in that range for individual samples.Thu, 09 Aug 2018 06:30:32 -0500https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/?comment=300157#post-id-300157Comment by pitosalas for <p>I would like to know the distance to the nearest obstacle in 8 wedges around the robot, i.e. N,E,W,S,NE,NW,SE,SW. My robot is a Turtlebot3 with just a Lidar but this question is general.</p>
<p>What I've done so far to see how far it gets is taking the mean value in each of the wedges and treating that as the distance of the nearest obstacle. So essentially for each scan I get back 8 numbers. I ignore values that are below or above the respective thresholds as well as infinity and NaN.</p>
<p>This is not good enough because of severe noise in the results of the scan (or bugs in my code. For now I am assuming noisy sensors.)</p>
<p>In consecutive scans I may get radically different vectors of 8 values (fake example data following:)</p>
<pre><code>[0.5, 0.4, 0.3, 1.2, 1.3, 1.4, 2.0]
[2.5, 1.4, 0.3, 1.3, 2.3, 1.4, 2.0]
</code></pre>
<p>Now remember that each of those numbers is a mean of 360/8 Lidar Readings. So there is some smoothing based on angle already happening. But it looks like I need smoothing over time too. Before I go reinventing the wheel I am looking for advice.</p>
<p>There's a nice laser_filters packages which looks very relevant but I am not sure its what I need.</p>
<h3>Followup</h3>
<p>An idea suggested below by <a href="/users/20134/geoff/">@Geoff</a> goes like this:</p>
<ol>
<li>Compute the vector of 8 values at time t. This can be done by for example taking the median +/- std dev or just the average of values in the second and third quartile.</li>
<li>Have time based filter by having a simple sliding window average or do something fancier with a Kaman filter.</li>
</ol>
<p>Thoughts?</p>
https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/?comment=300179#post-id-300179@peterblackerthe3d thanks that's very useful information. I will definitely double check with rviz as well as double check my algorithm.Thu, 09 Aug 2018 10:41:06 -0500https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/?comment=300179#post-id-300179Comment by PeteBlackerThe3rd for <p>I would like to know the distance to the nearest obstacle in 8 wedges around the robot, i.e. N,E,W,S,NE,NW,SE,SW. My robot is a Turtlebot3 with just a Lidar but this question is general.</p>
<p>What I've done so far to see how far it gets is taking the mean value in each of the wedges and treating that as the distance of the nearest obstacle. So essentially for each scan I get back 8 numbers. I ignore values that are below or above the respective thresholds as well as infinity and NaN.</p>
<p>This is not good enough because of severe noise in the results of the scan (or bugs in my code. For now I am assuming noisy sensors.)</p>
<p>In consecutive scans I may get radically different vectors of 8 values (fake example data following:)</p>
<pre><code>[0.5, 0.4, 0.3, 1.2, 1.3, 1.4, 2.0]
[2.5, 1.4, 0.3, 1.3, 2.3, 1.4, 2.0]
</code></pre>
<p>Now remember that each of those numbers is a mean of 360/8 Lidar Readings. So there is some smoothing based on angle already happening. But it looks like I need smoothing over time too. Before I go reinventing the wheel I am looking for advice.</p>
<p>There's a nice laser_filters packages which looks very relevant but I am not sure its what I need.</p>
<h3>Followup</h3>
<p>An idea suggested below by <a href="/users/20134/geoff/">@Geoff</a> goes like this:</p>
<ol>
<li>Compute the vector of 8 values at time t. This can be done by for example taking the median +/- std dev or just the average of values in the second and third quartile.</li>
<li>Have time based filter by having a simple sliding window average or do something fancier with a Kaman filter.</li>
</ol>
<p>Thoughts?</p>
https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/?comment=300156#post-id-300156Have you viewed the raw laser_scan message in RVIZ? If your taking mean averages of 45 degree sections and they're varying by over a meter then something really strange is going on!Thu, 09 Aug 2018 06:29:52 -0500https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/?comment=300156#post-id-300156Comment by Pujie for <p>I would like to know the distance to the nearest obstacle in 8 wedges around the robot, i.e. N,E,W,S,NE,NW,SE,SW. My robot is a Turtlebot3 with just a Lidar but this question is general.</p>
<p>What I've done so far to see how far it gets is taking the mean value in each of the wedges and treating that as the distance of the nearest obstacle. So essentially for each scan I get back 8 numbers. I ignore values that are below or above the respective thresholds as well as infinity and NaN.</p>
<p>This is not good enough because of severe noise in the results of the scan (or bugs in my code. For now I am assuming noisy sensors.)</p>
<p>In consecutive scans I may get radically different vectors of 8 values (fake example data following:)</p>
<pre><code>[0.5, 0.4, 0.3, 1.2, 1.3, 1.4, 2.0]
[2.5, 1.4, 0.3, 1.3, 2.3, 1.4, 2.0]
</code></pre>
<p>Now remember that each of those numbers is a mean of 360/8 Lidar Readings. So there is some smoothing based on angle already happening. But it looks like I need smoothing over time too. Before I go reinventing the wheel I am looking for advice.</p>
<p>There's a nice laser_filters packages which looks very relevant but I am not sure its what I need.</p>
<h3>Followup</h3>
<p>An idea suggested below by <a href="/users/20134/geoff/">@Geoff</a> goes like this:</p>
<ol>
<li>Compute the vector of 8 values at time t. This can be done by for example taking the median +/- std dev or just the average of values in the second and third quartile.</li>
<li>Have time based filter by having a simple sliding window average or do something fancier with a Kaman filter.</li>
</ol>
<p>Thoughts?</p>
https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/?comment=300115#post-id-300115Maybe you can search for [link text](http://wiki.ros.org/laser_geometry)Wed, 08 Aug 2018 23:46:28 -0500https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/?comment=300115#post-id-300115Answer by Geoff for <p>I would like to know the distance to the nearest obstacle in 8 wedges around the robot, i.e. N,E,W,S,NE,NW,SE,SW. My robot is a Turtlebot3 with just a Lidar but this question is general.</p>
<p>What I've done so far to see how far it gets is taking the mean value in each of the wedges and treating that as the distance of the nearest obstacle. So essentially for each scan I get back 8 numbers. I ignore values that are below or above the respective thresholds as well as infinity and NaN.</p>
<p>This is not good enough because of severe noise in the results of the scan (or bugs in my code. For now I am assuming noisy sensors.)</p>
<p>In consecutive scans I may get radically different vectors of 8 values (fake example data following:)</p>
<pre><code>[0.5, 0.4, 0.3, 1.2, 1.3, 1.4, 2.0]
[2.5, 1.4, 0.3, 1.3, 2.3, 1.4, 2.0]
</code></pre>
<p>Now remember that each of those numbers is a mean of 360/8 Lidar Readings. So there is some smoothing based on angle already happening. But it looks like I need smoothing over time too. Before I go reinventing the wheel I am looking for advice.</p>
<p>There's a nice laser_filters packages which looks very relevant but I am not sure its what I need.</p>
<h3>Followup</h3>
<p>An idea suggested below by <a href="/users/20134/geoff/">@Geoff</a> goes like this:</p>
<ol>
<li>Compute the vector of 8 values at time t. This can be done by for example taking the median +/- std dev or just the average of values in the second and third quartile.</li>
<li>Have time based filter by having a simple sliding window average or do something fancier with a Kaman filter.</li>
</ol>
<p>Thoughts?</p>
https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/?answer=300096#post-id-300096The `MedianFilter` and `MeanFilter` filters, which are used via the `LaserArrayFilter`, are quite useful and you could use them in place of doing your own averaging.
You can take care of the noise through various statistical methods ranging from the simple (average a bunch of values over time and/or space) to the hard-but-impresses-your-boss.
A simple approach that sometimes works is to calculate the centre of the range of distances in a wedge (i.e. `closest + (farthest - closest)/2`) and discard all values that are a certain distance away from that centre. It's kind of like a poor man's quartile filter. A more statistically accurate approach is to discard values that are a certain number of standard deviations from the mean, or to use only the second and third quartiles of the data, based on distance. Essentially what you are doing is trying to find a cluster of similar distances and using that as the "true" value. However, the larger the wedge, the less accurate these sorts of approaches become because you can easily end up ignoring smaller obstacles in the environment, or a corner that only appears in one small part of the wedge. These techniques are really better applied to a single range value over time rather than to a set of range values distributed over an angle.
The *really* fancy way to do something like this is to know the noise model of the sensor. This gives you a statistical probability for where the measurement value is likely to be for a given distance. Although noise models are more commonly used on single-value sensors over time (i.e. you can estimate the actual value by seeing how the measured noisy value jumps around the distribution of possible values), you can still use this information to weight each measurement that goes into the final value for each wedge - especially if you are filtering over time as well. You didn't say what sensor you are using, but since you are using the Turtlebot 3 I'm guessing that it's that [cheap little scanner that comes with it](http://www.robotis.us/360-laser-distance-sensor-lds-01-lidar/)? I doubt the maker provides a noise model for that one, unfortunately - and with that sensor I think it's almost certainly noise rather than bugs in your code causing you problems. You would have to construct one yourself by taking lots of careful measurements and accumulating data to process into a statistical model. That's probably a bit more work than is worth it for your task. :)
I think that give your sensor and your relatively simple goal, the best approach would be to filter the noise out either before or after calculating the value for the wedge. You can use a moving window of the data over time and calculate the value of the windowed data using something as simple as averaging each index over the time range. Or you can use a more accurate and advanced approach of applying a statistical filter that operates over time such as a Kalman filter (filtering a noisy signal to estimate the actual value is the canonical application of a Kalman filter). You would need to experiment to find out if you need to do this on the raw data from the sensor or if you can get away with doing your post processing after you calculate the value for each wedge. If you decide to use a Kalman filter then doing it on the sum of the wedge would be more computationally efficient, although the results may not be as good as doing it on each individual range measurement.
By the way, the `LaserScanRangeFilter` will help you with removing values that are below or above thresholds - although maybe it's overkill if you only have 8 values to check, not 1024.Wed, 08 Aug 2018 20:20:09 -0500https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/?answer=300096#post-id-300096Comment by pitosalas for <div class="snippet"><p>The <code>MedianFilter</code> and <code>MeanFilter</code> filters, which are used via the <code>LaserArrayFilter</code>, are quite useful and you could use them in place of doing your own averaging.</p>
<p>You can take care of the noise through various statistical methods ranging from the simple (average a bunch of values over time and/or space) to the hard-but-impresses-your-boss.</p>
<p>A simple approach that sometimes works is to calculate the centre of the range of distances in a wedge (i.e. <code>closest + (farthest - closest)/2</code>) and discard all values that are a certain distance away from that centre. It's kind of like a poor man's quartile filter. A more statistically accurate approach is to discard values that are a certain number of standard deviations from the mean, or to use only the second and third quartiles of the data, based on distance. Essentially what you are doing is trying to find a cluster of similar distances and using that as the "true" value. However, the larger the wedge, the less accurate these sorts of approaches become because you can easily end up ignoring smaller obstacles in the environment, or a corner that only appears in one small part of the wedge. These techniques are really better applied to a single range value over time rather than to a set of range values distributed over an angle.</p>
<p>The <em>really</em> fancy way to do something like this is to know the noise model of the sensor. This gives you a statistical probability for where the measurement value is likely to be for a given distance. Although noise models are more commonly used on single-value sensors over time (i.e. you can estimate the actual value by seeing how the measured noisy value jumps around the distribution of possible values), you can still use this information to weight each measurement that goes into the final value for each wedge - especially if you are filtering over time as well. You didn't say what sensor you are using, but since you are using the Turtlebot 3 I'm guessing that it's that <a href="http://www.robotis.us/360-laser-distance-sensor-lds-01-lidar/">cheap little scanner that comes with it</a>? I doubt the maker provides a noise model for that one, unfortunately - and with that sensor I think it's almost certainly noise rather than bugs in your code causing you problems. You would have to construct one yourself by taking lots of careful measurements and accumulating data to process into a statistical model. That's probably a bit more work than is worth it for your task. :)</p>
<p>I think that give your sensor and your relatively simple goal, the best approach would be to filter the noise out either before or after calculating the value for the wedge. You can use a moving window of the data over time and calculate the value of the windowed data using something as simple as averaging each index over the time range. Or you can use a more accurate and advanced approach of applying a statistical filter that operates over ...<span class="expander"> <a>(more)</a></span></p></div>https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/?comment=300178#post-id-300178Thanks for taking the time for such a detailed answer! As this comment doesn't have enough space, please see the original question for a restatement and follow up to your suggestion.Thu, 09 Aug 2018 10:35:25 -0500https://answers.ros.org/question/300094/getting-information-out-of-the-scan-topic/?comment=300178#post-id-300178