Ask Your Question
1

Streaming and visualizing large point clouds with rosbridge

asked 2021-09-16 03:43:54 -0600

n.dafinski gravatar image

updated 2021-09-16 03:48:51 -0600

Hey, so we've got a lidar sensor which produces a pretty large number of points. We're using that sensor for a SLAM algorithm and we want a way to visualize the whole map being generated by the SLAM in real-time ( with decay time like in rviz where I can keep the old points from the previous scans and increment and add to them ), but obviously filtered and downsampled a bit to make it more manageable to render. We're still debating whether to use a web app or a mobile app, for the mobile app the three best solutions I think I've come across so far would be Unity, Qt or Kivy. So I'm asking for a little bit of general advice, I guess, as to how to approach this, since we need to be able to handle a decent chunk of data being thrown into the app itself from ROS. I've tried the rosbridge_suite and with the very basic examples I was able to find I managed to get a cloud to visualize with a websocket and a tiny web app, but the performance wasn't great and it was eating a lot of CPU power on the client-side, I assume to process the incoming messages and render them? I did a little bit of research and found a few possible solutions to improve the performance like throttling the publisher to just publish the map less frequently, I saw some people trying to use different methods for message compression and serialization like BSON, and I also was debating if it's possible to visualize my map on the server-side in rviz and then send that to the client as a sort of 'screen-sharing' or stream so the client doesn't have to process all those points, but I just don't have all the knowledge necessary in this case to come up with the best solution, so that's why I'm here asking for your support! Thanks in advance

edit retag flag offensive close merge delete

2 Answers

Sort by » oldest newest most voted
0

answered 2021-10-27 15:28:26 -0600

jkovacich gravatar image

We're looking at doing something similar with large point clouds in real time, just curious if you landed on anything performant with rosbridge? Our use case has us using unity.

edit flag offensive delete link more

Comments

Hey, so far I'm still looking at getting good performance out of either using ros3djs to render the points on the client-side with rosbridge and roslib to make the connection, or using web_video_server to basically stream my RViz camera view but that makes it much harder to get that interactivity of moving the view window and paning/zooming which I definitely would like to keep. I haven't managed to get either of the options to a level which I'm comfortable and happy with, so entirely different approaches are not of the question for me yet. Can you tell me a bit more about how you plan to use Unity to solve your problem?

n.dafinski gravatar image n.dafinski  ( 2021-10-29 09:29:06 -0600 )edit

Sure. So in our case we already have a visualization tool and we're switching middleware. Our sensors are emitting point clouds using ROS nodes in c++. We render large point clouds and markers in real time, and I'm looking to integrate ROS messages, 60k points/frame, 10 frames a second or so for the main topic and we need to be able to support N topics and receive/visualize them in real time. When I attempted to connect using rosbridge and json serialization I was getting a message every 5 or so seconds, when it needed to be 10x/s. I assumed BSON would be a little quicker but likely not quick enough for our use case.

I'm currently exploring ROS-TCP-Connector instead since it seems to make more use of the internal binary structures instead of JSON serialization. If that doesn't pan out our options are ...(more)

jkovacich gravatar image jkovacich  ( 2021-10-29 09:42:47 -0600 )edit

I was also considering Unity at first and might still look into it if my current options don't pan out, I remember reading a bit about ROS-TCP-Connector and thinking it might be worth checking out. If you don't mind me asking what tools/technologies are you using to render such huge points clouds? I've been researching a lot for some kind of tool that can do that on the web but it seems like most of the ones that exist are proprietary or maybe have nothing to do with ROS at all

n.dafinski gravatar image n.dafinski  ( 2021-10-29 10:15:36 -0600 )edit

In our case we're making a desktop application, so we're using Unity's MeshRenderer/MeshFilter to draw everything. Get a new cloud of points, clear the old one, assign the new one, rinse, repeat.

jkovacich gravatar image jkovacich  ( 2021-10-29 12:35:26 -0600 )edit
0

answered 2021-09-16 08:33:32 -0600

There is an interesting fairly recent published paper by Antonia Zea: A ROS Visualization App for Mobile Devices, https://arxiv.org/pdf/2008.12725.pdf. As you mention is a challenging area of research due to limitations with tools.

They also have open source the work they have done with their App in Unity: https://github.com/KIT-ISAS/iviz/tree...

Hope this helps. I will be following the question, very interesting topic, thank you for raising.

edit flag offensive delete link more

Comments

Hey, I've had a look at iviz yea, it looks really interesting but unfortunately I wasn't able to get it working for me on Desktop, I might try mobile tomorrow

n.dafinski gravatar image n.dafinski  ( 2021-09-16 10:07:08 -0600 )edit

but I’m still not sure how reliable Unity would be for a larger scale application and real-world use

n.dafinski gravatar image n.dafinski  ( 2021-09-16 10:29:20 -0600 )edit

Unity is deployed extensively in gaming at large scale in mobile, it's very stable.

osilva gravatar image osilva  ( 2021-09-16 10:46:34 -0600 )edit

yep, I am a bit familiar with Unity but I was more referring to its integration with ROS and how reliable that is, cause I still haven't seen many examples of real-world apps used in industry built on ROS, and even fewer using Unity, but of course I'm not saying it's not possible, I'm just not sure but combining ROS with a game engine like Unity sounds like a really interesting pairing

n.dafinski gravatar image n.dafinski  ( 2021-09-16 11:18:25 -0600 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2021-09-16 03:43:54 -0600

Seen: 44 times

Last updated: Sep 16 '21