Best Practices for Publishing Lots of Related Data [closed]
I am developing a system using ROS2 Foxy on Ubuntu 20.04 that interfaces with a Allen Bradley ControlLogix PLC, speaking EtherNet/IP. We pull data from it at 20Hz, and there is a large amount of information that we are pulling. The code is mostly Python, but there are a few C++ components.
We currently capture this data and publish it on a topic as yaml serialised String, as the structure of the data made it prohibitive to build a Message file. The format has become fairly stable, and there is a need/wish to allow remapping of topics to swap in some of the data for testing etc.
What is the best practice for this? If I make a single topic with a custom message format, the code will be less portable, and there wil be a lot of data being transported unnecessarily. If I go with individual topics, there will be potentially hundreds. My last thought is to go with an inbetween solution, and publish groups of related data. However, I believe the best practice is to avoid custom messages when you can, and this could potentially require many custom messages to group different things. There are many different pieces of equipment attached to the PLC.
I've come across some other posts "Many Topics" was the most relevant. However, the tradeoffs mentioned by the answer there focus on timestamping, rather than broader best practice.
What kind of tradeoff makes the most sense in peoples experience?
Thank you in advance.
Update
In the end, we managed to rearrange a number of UDTs so that the most import data could be published as a set of individual topics, with the remaining slowly being converted from the monolithic yaml to sets of nested keyword lists, as in @lucasw's answer.
We felt this gave the best tradeoff in performance, sanity (only one serialiser on the wire) and human viewability (keyword list instead of just lists, where you would need a separate table at all times to see what the piece of data is, and then you would need to keep those in sync)
Could you specify the specific PLC you are using? It could be that ROS package already exists for it. If a package doesn't exist, the specification should help others understand the scope of your problem.
@kscottz, It is an Allen Bradley ControlLogix PLC, speaking EtherNet/IP. We have written a custom node for it at this time. Looking around, I was not able to find ros2 PLC node that could work in our environment. Thank you for the help!