Ask Your Question

Best Practices for Publishing Lots of Related Data [closed]

asked 2022-02-15 18:15:21 -0500

alluring-mushroom gravatar image

updated 2022-03-08 02:38:03 -0500

I am developing a system using ROS2 Foxy on Ubuntu 20.04 that interfaces with a Allen Bradley ControlLogix PLC, speaking EtherNet/IP. We pull data from it at 20Hz, and there is a large amount of information that we are pulling. The code is mostly Python, but there are a few C++ components.

We currently capture this data and publish it on a topic as yaml serialised String, as the structure of the data made it prohibitive to build a Message file. The format has become fairly stable, and there is a need/wish to allow remapping of topics to swap in some of the data for testing etc.

What is the best practice for this? If I make a single topic with a custom message format, the code will be less portable, and there wil be a lot of data being transported unnecessarily. If I go with individual topics, there will be potentially hundreds. My last thought is to go with an inbetween solution, and publish groups of related data. However, I believe the best practice is to avoid custom messages when you can, and this could potentially require many custom messages to group different things. There are many different pieces of equipment attached to the PLC.

I've come across some other posts "Many Topics" was the most relevant. However, the tradeoffs mentioned by the answer there focus on timestamping, rather than broader best practice.

What kind of tradeoff makes the most sense in peoples experience?

Thank you in advance.


In the end, we managed to rearrange a number of UDTs so that the most import data could be published as a set of individual topics, with the remaining slowly being converted from the monolithic yaml to sets of nested keyword lists, as in @lucasw's answer.

We felt this gave the best tradeoff in performance, sanity (only one serialiser on the wire) and human viewability (keyword list instead of just lists, where you would need a separate table at all times to see what the piece of data is, and then you would need to keep those in sync)

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by alluring-mushroom
close date 2022-03-08 02:38:46.816985



Could you specify the specific PLC you are using? It could be that ROS package already exists for it. If a package doesn't exist, the specification should help others understand the scope of your problem.

kscottz gravatar image kscottz  ( 2022-02-17 16:35:59 -0500 )edit

@kscottz, It is an Allen Bradley ControlLogix PLC, speaking EtherNet/IP. We have written a custom node for it at this time. Looking around, I was not able to find ros2 PLC node that could work in our environment. Thank you for the help!

alluring-mushroom gravatar image alluring-mushroom  ( 2022-02-20 18:20:42 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted

answered 2022-02-18 18:55:37 -0500

lucasw gravatar image

updated 2022-02-18 18:56:20 -0500

I had 10s instead of 100s of different topics but I recently did something similar where a received message was parsed and then published onto individual topics (I used marti_common_msgs/Float32Stamped because I wanted the reception time captured). It pollutes the topic list some but it's very easy to plot any of the values, which isn't going to work well with yaml.

Then there had to be a second layer to subscribe to some of those topics to turn into JointStates- which added a little latency (but it's all ros1 tcp on localhost with messages that are a handful of bytes, so not that much). Most of the rest had no subscribers other than rosbag. So if you have groups of data that map to existing message types those would be good to publish out, then they will be easy to visualize in rviz or use in other off the shelf nodes- like groups of xyz into Points or Vectors. Anything you have a real subscriber for right now, where different actions will be taken automatically based on the value of the data, or someone needs to monitor and make live decisions based on plots or 3D visualization- those deserve more consideration for what kind of message they go in, everything else could be swept into a big generic array message for recording.

From ros1 the dynamic_reconfigure/Config message could be a model to use for a big generic array- there are arrays of different types along with string names for each array element, though it seems wasteful to resend the same string names every message unless message-to-message there are changes in which name is in which index of the array (it ought to be more efficient than sending a bunch of doubles as strings in yaml though):

$ rosmsg show dynamic_reconfigure/Config
dynamic_reconfigure/BoolParameter[] bools
  string name
  bool value
dynamic_reconfigure/IntParameter[] ints
  string name
  int32 value
dynamic_reconfigure/StrParameter[] strs
  string name
  string value
dynamic_reconfigure/DoubleParameter[] doubles
  string name
  float64 value
dynamic_reconfigure/GroupState[] groups
  string name
  bool state
  int32 id
  int32 parent
edit flag offensive delete link more



Hello, thank you for the answer and your time! I've been considering using something similar to the dynamic_reconfigure/Config option. The reason I like this is I would have a very large array of data, and when plotting/referencing it, it would be very easy to be looking at the wrong value (as a human) if they weren't labelled in some way. All of the communication is local, so like you say, the overhead shouldn't be too extreme for including strings. Finally, we have managed to isolate around 30 of the most important messages to publish individually (rather than nested as they would otherwise have been) Its only one step away from the yaml, but so far it seems to be the best idea.

alluring-mushroom gravatar image alluring-mushroom  ( 2022-02-20 17:52:56 -0500 )edit

Question Tools



Asked: 2022-02-15 18:12:50 -0500

Seen: 92 times

Last updated: Mar 08