Ask Your Question
0

ROS2 serialization ConnextStaticSerializedData [closed]

asked 2019-09-09 03:49:15 -0500

qootec gravatar image

Environment:

  • ROS2 Dashing Patch 2
  • Windows 10
  • messages based on msg files (no idl yet, since I can't seem that to get that operational)
  • RTI Connext DDS 5.3.1

Reproduction:

  • Create a custom message type and a Publisher & Subscriber for it.
  • Start your nodes

Expected behavior:

  • They communicate - ok they do.
  • Their type is observable through RTI's Admin Console

Observed behavior:

  • The topic's type is not observable through RTI's Admin Console. Instead, Admin Console reports the type as "ConnextStaticSerializedData".
  • Consequentially, the Admin Console shows shows only partial matches (conflicting data type). image description
  • Even loading data types from XML (supported by RTI Admin Console) is not accepted (could be my mistake).

Question(s):

  • Does this mean that ROS2 is using non-standard serialisation? Looking at the code it seems that serialisation happens independently in the ROS2 RMW layer and is then submitted as a serialized packet to a topic typed by ConnextStaticSerializedData, whatever the original type. This happens for custom types, but e.g. not for the rq/…/list_parametersRequest and other built-in types.
  • Any way to avoid this behavior? It is quite annoying from an interoperability point of view (Admin console, but also other non ROS2 DDS-based applications).
  • Does the behavior change when I would use IDL instead of msg files? I didn't find a way to get that working although it should work on Dashing.

Thanks, Johan

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by qootec
close date 2019-09-13 03:02:39.852260

1 Answer

Sort by » oldest newest most voted
1

answered 2019-09-09 14:04:52 -0500

Karsten gravatar image

tldr - Here are my answers to your questions, I'll go in a bit more details after

Does this mean that ROS2 is using non-standard serialisation? Looking at the code it seems that serialisation happens independently in the ROS2 RMW layer and is then submitted as a serialized packet to a topic typed by ConnextStaticSerializedData, whatever the original type. This happens for custom types, but e.g. not for the rq/…/list_parametersRequest and other built-in types.

We are using a workaround to access the serialized code directly without deserializing to write it e.g. in rosbags. RTI Connext doesn't provide this functionality out of the box. That's why we implemented a way of type masquerading to do so - it's not really that, but it's somewhat similar. It was only implemented for topics to make pub/sub work. Therefore, you don't see that on services.

Any way to avoid this behavior? It is quite annoying from an interoperability point of view (Admin console, but also other non ROS2 DDS-based applications).

We are aware of the mismatch with RTI's Admin Console, but AFAICT there is no workaround. I am not quite sure if I understand your point on non ROS2 DDS-based applications. Our workaround is safe to use between multiple DDS vendors and is wire compatible (we run quite a few cross talking tests on the ROS2 buildfarm, e.g. Connext and FastRTPS).

Does the behavior change when I would use IDL instead of msg files? I didn't find a way to get that working although it should work on Dashing.

No. The message generation step is independent from the way we send it over the wire.


The change got introduced as a prerequisite for having Connext support rosbags. In order to speed up the write process, it's kind of necessary to obtain an incoming message without deserializing it first, then re-serialize it and write it to disk. Whereas FastRTPS made it quite easy to get access to the buffer with the serialized (CDR) data, there was no straight forward way to do so in Connext.

After discussing with RTI, we've developed that workaround together which essentially publishes a sequence of octets (shown as ConnextStaticSerializedData) which holds the CDR data. We had to patch the API of Connext in order to support that. The changes and more patch files can be found here: https://github.com/ros2/rmw_connext/b...

We declare the type of this message however as the actual data type of the message, e.g. std_msgs::msg::Stringto be wire compatible. https://github.com/ros2/rmw_connext/b...

Needless to say, we did the same workaround for subscribers.

The patch files applied are only compatible for Connext 5.3.1. These patch files have to be modified to match Connext 6.0.0, which we haven't done yet. The patched code is used for publishing in any case, whether you publish serialized code or ROS2 message instances. We could technically ... (more)

edit flag offensive delete link more

Comments

Hi Karsten, thanks for your extensive answer.

(from my question) It is quite annoying from an interoperability point of view (Admin console, but also other non ROS2 DDS-based applications).

This was based on an assumption of mine, but I guess I assumed wrongly… I thought that if Admin Console has a problem, other pure DDS applications would also run into the same issue, certainly in case they would be based on DDS Dynamic Data.

(from your reply) We declare the type of this message however as the actual data type of the message, e.g. std_msgs::msg::Stringto be wire compatible.

It seems that my DDS knowledge is insufficient to understand that you can send the correct metadata through the (patched) RTI API, but that the receiving application can still get confused about the topic's type. I would expect that the metadata is the only truth about a topic ...(more)

qootec gravatar imageqootec ( 2019-09-10 08:47:55 -0500 )edit

Question Tools

2 followers

Stats

Asked: 2019-09-09 03:49:15 -0500

Seen: 14 times

Last updated: Sep 09