Revision history [back]

Unfortunately, I believe this is by design.

See wiki/msg, specifically the parts about how Python 2 and Python 3 treat string and uint8[] fields.

It's essentially as you already described: in Python 3, a string field will be mapped onto bytes. In Python 2, it'll be str.

Pedantic, but I believe this is the problem:

I have to send binary String messages via the ROS message bus

According to wiki/msg again, string fields are intended to contain ascii strings, not binary data. Those two are not the same, both semantically and structurally (yes, they both typically occupy a contiguous piece of memory, and both are byte/char based, but that doesn't make them identical).

The violation of the semantics of string you propose then causes the problems with the code on different versions of ROS: it just happened to work prior to Noetic because Python 2 was very lax in this regard. But based on the semantics and the field type description, the developers were justified in fixing it in Noetic (they were also forced by the Python 3 migration of course).

Note: this is not a snooty answer trying to put blame on you. I just wanted to clarify for future readers why caring about semantics of ROS message fields (and messages in general) is important. Even if "it fits", it does not mean it's proper use.

Unfortunately, I believe this is by design.

See wiki/msg, specifically the parts about how Python 2 and Python 3 treat string and uint8[] fields.

It's essentially as you already described: in Python 3, a string field will be mapped onto bytes. In Python 2, it'll be str.

Pedantic, but I believe this is the problem:

I have to send binary String messages via the ROS message bus

According to wiki/msg again, string fields are intended to contain ascii strings, not binary data. Those two are not the same, both semantically and structurally (yes, they both typically occupy a contiguous piece of memory, and both are byte/char based, but that doesn't make them identical).

The violation of the semantics of string you propose then causes the problems with the code on different versions of ROS: it just happened to work prior to Noetic because Python 2 was very lax in this regard. But based on the semantics and the field type description, the developers were justified in fixing it in Noetic (they were also forced by the Python 3 migration of course).

Note: this is not a snooty answer trying to put blame on you. I just wanted to clarify for future readers why caring about semantics of ROS message fields (and messages in general) is important. Even if "it fits", it does not mean it's proper use.

Unfortunately, I believe this is by design.

See wiki/msg, specifically the parts about how Python 2 and Python 3 treat string and uint8[] fields.

It's essentially as you already described: in Python 3, a string field will be mapped onto bytes. In Python 2, it'll be str.

Pedantic, but I believe this is the problem:

I have to send binary String messages via the ROS message bus

According to wiki/msg again, string fields are intended to contain ascii strings, not binary data. Those two are not the same, both semantically and structurally (yes, they both typically occupy a contiguous piece of memory, and both are byte/char based, but that doesn't make them identical).

The violation of the semantics of string you propose then causes the problems with the code on different versions of ROS: it just happened to work prior to Noetic because Python 2 was very lax in this regard. But based on the semantics and the field type description, the developers were justified in fixing it in Noetic (they were also forced by the Python 3 migration of course).

Note: this is not a snooty answer trying to put blame on you. I just wanted to clarify for future readers why caring about semantics of ROS message fields (and messages in general) is important. Even if "it fits", it does not mean it's proper use.

Is there some way to work around that without modifying/destroying the binary data itself? Decoding the binary data first so that ROS can encode it again unfortunately does modify it.

if you really want to send binary data in string fields, I believe the only proper way would be to first encode it as Base64, and then code it again in all receivers.

It would still violate semantics, but it would let you use a string for something which is technically not an ascii string.

Unfortunately, I believe this is by design.

See wiki/msg, specifically the parts about how Python 2 and Python 3 treat string and uint8[] fields.

It's essentially as you already described: in Python 3, a string field will be mapped onto bytes. In Python 2, it'll be str.

Pedantic, but I believe this is the problem:

I have to send binary String messages via the ROS message bus

According to wiki/msg again, string fields are intended to contain ascii strings, not binary data. Those two are not the same, both semantically and structurally (yes, they both typically occupy a contiguous piece of memory, and both are byte/char based, but that doesn't make them identical).

The violation of the semantics of string you propose then causes the problems with the code on different versions of ROS: it just happened to work prior to Noetic because Python 2 was very lax in this regard. But based on the semantics and the field type description, the developers were justified in fixing it in Noetic (they were also forced by the Python 3 migration of course).

Note: this is not a snooty answer trying to put blame on you. I just wanted to clarify for future readers why caring about semantics of ROS message fields (and semantics of messages in general) is important. Even if "it fits", it does not mean it's proper use.use. Abusing message types can lead to situations where your assumption something will work is broken by a maintainer.

Is there some way to work around that without modifying/destroying the binary data itself? Decoding the binary data first so that ROS can encode it again unfortunately does modify it.

if you really want to send binary data in string fields, I believe the only proper way would be to first encode it as Base64, and then code it again in all receivers.

It would still violate semantics, but it would let you use a string for something which is technically not an ascii string.

It would of course not help with this:

would add massive overhead and break compatibility with ROS Melodic clients and rosbags.

Unfortunately, I believe this is by design.

See wiki/msg, specifically the parts about how Python 2 and Python 3 treat string and uint8[] fields.

It's essentially as you already described: in Python 3, a string field will be mapped onto bytes. In Python 2, it'll be str.

Pedantic, but I believe this is the problem:

I have to send binary String messages via the ROS message bus

According to wiki/msg again, string fields are intended to contain ascii strings, not binary data. Those two are not the same, both semantically and structurally (yes, they both typically occupy a contiguous piece of memory, and both are byte/char based, but that doesn't make them identical).

The violation of the semantics of string you propose then causes the problems with the code on different versions of ROS: it just happened to work prior to Noetic because Python 2 was very lax in this regard. But based on the semantics and the field type description, the developers were justified in fixing it in Noetic (they were also forced by the Python 3 migration of course).

Note: this is not a snooty answer trying to put blame on you. I just wanted to clarify for future readers why caring about semantics of ROS message fields (and semantics of messages in general) is important. Even if "it fits", it does not mean it's proper use. Abusing message types can lead to situations where your assumption something will work is broken by a maintainer.

Is there some way to work around that without modifying/destroying the binary data itself? Decoding the binary data first so that ROS can encode it again unfortunately does modify it.

if you really want to send binary data in string fields, I believe the only proper way would be to first encode it as Base64, and then code decode it again in all receivers.

It would still violate semantics, but it would let you use a string for something which is technically not an ascii string.

It would of course not help with this:

would add massive overhead and break compatibility with ROS Melodic clients and rosbags.