ROS1 vs ROS2 serialization

asked 2021-02-15 14:18:15 -0500

Ariel gravatar image

Hello,

I am serializing the following msg:

string a
uint8 b
uint8[] c

I am comparing the serialization between ROS1 and ROS2 and I don't know what are the "extra" bytes I get with ROS2:

Serialized ROS1 message: 11 00 00 00 03 00 00 00 41 42 43 0A 05 00 00 00 0B 0C 0D 0E 0F
Serialized ROS2 message: 19 00 00 00 00 01 00 00 04 00 00 00 41 42 43 00 0A 7f 00 00 05 00 00 00 0B 0C 0D 0E 0E

I am also printing the length of the messages (first 4 bytes). Some differences I see:

  1. ROS2 adds 00 01 00 00 at the beginning of the message. What are these 4 bytes represent? Are they related to the message?
  2. For mytype.a it ends with 00 in ROS2, is it something related to it or I made a mistake somewhere?
  3. What are the 3 bytes before the length of mytype.c ? (7f 00 00)?

Here is the code for ROS1:

#include "ros/ros.h"
#include "serialization/mytypes.h"

#include <sstream>
#include <iostream>
#include <fstream>
#include <boost/format.hpp>

namespace ser = ros::serialization;
using namespace std;

serialization::mytypes create_serialization_mytypes();

serialization::mytypes create_serialization_mytypes() {
    serialization::mytypes result;

    result.a = "ABC";
    result.b = 10;
    for(int i=0; i<5; i++){
        result.c.push_back(11+i);
    }
    return result;
}

int main(int argc, char **argv)
{
    ros::init(argc, argv, "node_publisher");
    ros::NodeHandle n;

    serialization::mytypes mytypes;
    mytypes=create_serialization_mytypes();

    uint32_t serial_size = ros::serialization::serializationLength(mytypes);
    boost::shared_array<uint8_t> buffer(new uint8_t[serial_size]);

    ser::OStream stream(buffer.get(), serial_size);
    ser::serialize(stream, mytypes);
    std::cout << "Serialized ROS1 message: ";
    std::cout << std::uppercase << setfill('0') << setw(2) << hex << +(serial_size&0xFF) << " ";
    std::cout << std::uppercase << setfill('0') << setw(2) << hex << +(serial_size>>8&0xFF) << " ";
    std::cout << std::uppercase << setfill('0') << setw(2) << hex << +(serial_size>>16&0xFF) << " ";
    std::cout << std::uppercase << setfill('0') << setw(2) << hex << +(serial_size>>24&0xFF) << " ";
    for(int i=0; i<serial_size; i++){
        std::cout << std::uppercase << setfill('0') << setw(2) << hex << +(buffer[i]) << " ";
    }
    std::cout << endl;

  return 0;
}

And the code for ROS2:

#include <chrono>
#include <memory>

#include "rclcpp/rclcpp.hpp"

#include "rclcpp/serialization.hpp"
#include "serialization/msg/mytypes.hpp"

using namespace std::chrono_literals;

class MinimalPublisher : public rclcpp::Node
{
public:

    serialization::msg::Mytypes create_serialization_mytypes() {
        serialization::msg::Mytypes result;

        result.a = "ABC";
        result.b = 10;
        for(int i=0; i<5; i++){
            result.c.push_back(11+i);
        }
        return result;
    }
  MinimalPublisher()
  : Node("minimal_publisher"), count_(0)
  {
    publisher_ = this->create_publisher<serialization::msg::Mytypes>("topic", 10);    // CHANGE

    // printf("Finished\n");
    timer_ = this->create_wall_timer(
      500ms, std::bind(&MinimalPublisher::timer_callback, this));
  }

private:
  void timer_callback()
  {
    auto message = serialization::msg::Mytypes();
    message = create_serialization_mytypes();

    auto message_header_length = 8u;
    auto message_payload_length = static_cast<size_t>(sizeof(message));
    serialized_msg.reserve(message_header_length + message_payload_length);

    static rclcpp::Serialization<serialization::msg::Mytypes> serializer;
    //serializer.serialize_message(message.get(), &serialized_msg);
    serializer.serialize_message(&message, &serialized_msg);

    printf("Serialized ROS2 message: ");
    printf("%02x ", (serialized_msg.size()&0xFF));
    printf("%02x ", (serialized_msg.size()>>8&0xFF));
    printf ...
(more)
edit retag flag offensive close merge delete

Comments

As discussed in #q371410, ROS 2 uses DDS as the default middleware. DDS implementations typically use CDR (Common Data Representation) as the serialisation format.

You probably want to look into the relevant standards. The OMG standardises just about everything to do with DDS.

For ROS 2 specifically, in Foxy, the default middleware is Fast-DDS, which uses Fast-CDR.

Wireshark should be able to dissect the common RTPS implementations as well.

gvdhoorn gravatar image gvdhoorn  ( 2021-02-15 14:26:38 -0500 )edit

It is possible to dissect it with wireshark. However, I haven't seen something like http://wiki.ros.org/ROS/Connection%20Header with a similar, simple and straight forward explanation so far

Ariel gravatar image Ariel  ( 2021-02-15 15:09:30 -0500 )edit

Could you describe what it is you're trying to do? Are you writing a custom client library and want/need to deserialise message payload yourself, or is this purely academic interest? If the former, reusing an existing implementation may be more efficient. If the latter, you may want to refer to the standards surrounding CDR, XCDR(2) and dynamic datatypes et al.

However, I haven't seen something like http://wiki.ros.org/ROS/Connection%20... with a similar, simple and straight forward explanation so far

that's probably because it's not so simple and it's not ROS 2 specific any more, in contrast to the ROS 1 implementations. Part of the reason to rely on a standardised middleware is to be able to delegate the work of implementing, maintaining and documenting everything yourself.

Perhaps eProsima/Micro-CDR could help as well. It's a fairly low-level and ...(more)

gvdhoorn gravatar image gvdhoorn  ( 2021-02-16 02:47:05 -0500 )edit

Pedantic, but:

Some differences I see:

  1. ROS2 adds 00 01 00 00 at the beginning of the message.

actually, that's not "ROS 2", but whichever middleware you have configured for your particular installation.

by default, that'll be Fast-DDS with Fast-RTPS and their implementation of CDR.

But it could be Intel's DPS, or Continental's eCAL.

gvdhoorn gravatar image gvdhoorn  ( 2021-02-16 02:52:05 -0500 )edit

Could you describe what it is you're trying to do? Are you writing a custom client library and want/need to deserialise message payload yourself, or is this purely academic interest?

Yes, it is a sort of custom client library and I need to serialize/deseralize so that is why I am using the provided functions. It is not a completely client library. I am only concern about the serialization/deserialization. So, how it is being done already as the code I have it is perfect. My issue is to understand the "extra" bytes generated by the "ROS2" code vs the ROS1 one.

Ariel gravatar image Ariel  ( 2021-02-16 03:02:09 -0500 )edit

Yes, it is a sort of custom client library and I need to serialize/deseralize

then I would suggest using an existing CDR implementation instead of trying to reverse engineer it yourself.

So, how it is being done already as the code I have it is perfect

well, you seem to be going in the direction of implementing a CDR (de)serialisation library yourself. I wouldn't call that perfect. Personally I'd reuse an existing implementation.

gvdhoorn gravatar image gvdhoorn  ( 2021-02-16 03:06:05 -0500 )edit

I also don't believe you can assume DDS (with CDR) will be used necessarily.

I haven't checked, but it could well be that other middlewares will use a different serialisation scheme altogether.

See #q369220 for a Q&A with links to a few different RMW implementations. As you can see there are quite a few.

And see design/ROS 2 middleware interface for information on how the RMW abstraction layer works.

gvdhoorn gravatar image gvdhoorn  ( 2021-02-16 03:07:53 -0500 )edit