# smach introspection server fails in electric

Hi everyone,

I am using smach in electric with the introspection server and the smach_viewer.py.

In electric the smach_viewer.py does not work for me (it just shows the machine but does not update in which state the machine is) and the script running the introspection server periodically shows the following exception:

Exception in thread server_name:status_publisher:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 505, in run
self.__target(*self.__args, **self.__kwargs)
File "/opt/ros/electric/stacks/executive_smach/smach_ros/src/smach_ros/introspection.py", line 160, in _status_pub_loop
self._publish_status('HEARTBEAT')
File "/opt/ros/electric/stacks/executive_smach/smach_ros/src/smach_ros/introspection.py", line 225, in _publish_status
self._status_pub.publish(state_msg)
File "/opt/ros/electric/stacks/ros_comm/clients/rospy/src/rospy/topics.py", line 695, in publish
self.impl.publish(data)
File "/opt/ros/electric/stacks/ros_comm/clients/rospy/src/rospy/topics.py", line 872, in publish
serialize_message(b, self.seq, message)
File "/opt/ros/electric/stacks/ros_comm/clients/rospy/src/rospy/msg.py", line 151, in serialize_message
msg.serialize(b)
File "/opt/ros/electric/stacks/executive_smach/smach_msgs/src/smach_msgs/msg/_SmachContainerStatus.py", line 120, in serialize
buff.write(struct.pack('<I%ss'%length, length, _x.encode()))
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)


Without the introspection sever or when running smach from diamondback the exception does not show up and everything works as expected. Does anyone have a clue what the problem is?

I used the script "examples/user_data2.py" from the smach_tutorials package and added the introspection server as follows:

--- user_data2.py   2011-08-31 09:41:46.108424577 +0200
+++ user_data_sis.py    2011-08-31 10:13:34.652678420 +0200
@@ -59,7 +59,11 @@

# Execute SMACH plan
+   sis = smach_ros.IntrospectionServer('server_name', sm, '/SM_ROOT')
+   sis.start()
outcome = sm.execute()
+    rospy.spin()
+    sis.stop()

if __name__ == '__main__':


edit retag close merge delete

This looks like somewhere there's a unicode issues. Is there any unicode used in your smach state machine states? 0x80 is not a valid ascii code as the backtrace suggests.
( 2011-09-29 05:25:36 -0500 )edit

Sort by » oldest newest most voted

I suspect this is a regression caused by a Python 3k compatibility patch in the message generators. I have backed this out in r15301, but it will take awhile to get this fix deployed as it will required a full deb rebuild.

more

If so, it looks to me like the Python 3k compatibility is just more restrictive. Really the pickled user data shouldn't be "string" it should be some byte array.
( 2011-10-28 06:01:55 -0500 )edit
realistically, for true py3k compatibility, the message generator needs to be rewritten to start using "bytes" and use the distinctions that the Python 2.6+ APIs now provide. This is a significant undertaking, so for now, backing out the Py3k patch.
( 2011-10-28 06:07:22 -0500 )edit

Hi,

i just rechecked the problem with the ascii-encoding and it´s actually less a problem than i thought it is. It was actually bug in my patch. On

"struct.pack() now only allows bytes for the s string pack code. Formerly, it would accept text arguments and implicitly encode them to bytes using UTF-8. This was problematic because it made assumptions about the correct encoding and because a variable-length encoding can fail when writing to fixed length segment of a structure."

So since Python3.2 struct.pack did an implicit encoding of string-data using UTF8-encoding which since Python3.2 has to be done explicitly. In my patch i therefore added "s.encode()" which uses the python standard string encoding (which apparently was set to "ascii" at Manuels computer) instead of "utf-8" like it did before. So changing this to s.encode("utf-8") should produce the same behaviour in Python2.6 and also work with Python3.2 even if your standard string-encoding is set to "ascii".

@Ken: Do you want me to send a patch for this issue?

Cheers

Michi

more

line 643 in genpy.py should look like this: yield pack2("'<I%ss'%length", "length, %s.encode("utf-8")"%var)
( 2011-11-08 22:16:00 -0500 )edit
str.encode() is stateful and can't be repeated for strings previously encoded -- following up in e-mail to discuss more.
( 2011-11-09 04:23:31 -0500 )edit

Hey guys,

sorry for my late reply. I guess this is indeed caused by trying to introduce partial Python3-compatibility into ROS. The "encode()" and "decode()" parts had to be added because of the new string representation in Python 3 (Unicode strings and bytes for data instead of unicode and 8-bit strings, see http://docs.python.org/release/3.0.1/whatsnew/3.0.html#text-vs-data-instead-of-unicode-vs-8-bit).

However it seems like the _x.encode() uses the current standard string encoding to get from string to bytes which seems to be 'ascii' in your case while on my computer it is 'utf-8'. This could be the cause why Wim could not reproduce the problem. To test this, could you set your default string encoding to utf-8 and check if you still get the error?

You can set your default encoding using:

sys.setdefaultencoding('utf-8')

I am not familiar with the smach viewer, but i would be interested what kind of data you are sending SmachContainerStatus-messages...

If this works, there is the possibility that we force the encoding to be utf-8 in the messages... of course, we would have to set the decode-functions to utf-8 too....

Hope this helps and sorry again for the late reply...

Cheers

Michi

more

I posted this to ros-users as well, but I'll add it here for completeness.

It appears that in Electric, rospy enforces that strings in ROS messages be ASCII strings, so the output of the python pickle module can't be encoded. To fix this we should be able to just change the "local_data" field in https://kforge.ros.org/smach/executive_smach/file/d79964e4c22f/smach_msgs/msg/SmachContainerStatus.msg">SmachContainerStatus from:

12 # A pickled user data structure
13 string local_data


to:

12 # A pickled user data structure
13 uint8[] local_data


I don't currently have the time to test this fix, but I'm pretty sure this is the problem.

more

actually, should be uint8[]. "byte" is deprecated and shouldn't be used (it's been hard to actually remove due to widespread use and md5sum compatibility)
( 2011-10-28 06:08:20 -0500 )edit
fixed in edit
( 2011-10-28 09:56:21 -0500 )edit

Hi all, I'm having the same trouble that Sebastian... Has anyone solved it?

more

I'm also in this trouble. This occurs when run the patched user_data2.py and subscribing "/server_name/smach/container_status" topic.

File "/opt/ros/electric/stacks/executive_smach/smach_msgs/src/smach_msgs/msg/_SmachContainerStatus.py", line 120, in serialize
buff.write(struct.pack('<I%ss'%length, length, _x.encode()))
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)


When I modified the introspection.py, the error does not occur.

diff -r 46cddfcd1bf3 smach_ros/src/smach_ros/introspection.py
--- a/smach_ros/src/smach_ros/introspection.py  Tue Aug 02 10:31:19 2011 -0700
+++ b/smach_ros/src/smach_ros/introspection.py  Thu Sep 29 22:31:48 2011 +0900
@@ -219,7 +219,7 @@
path,
self._container.get_initial_states(),
self._container.get_active_states(),
-                    pickle.dumps(self._container.userdata._data,2),
+                    "",
info_str)
# Publish message
self._status_pub.publish(state_msg)

more

This is throwing the baby out with the bathwater. It's removing the part of the status message that includes the user data.
( 2011-10-28 05:50:00 -0500 )edit

I haven't been able to reproduce this problem on the latest Electric Debians. Are you sure you're not mixing Electric and Diamondback? It's possible that the Electric viewer cannot connect to a Diamondback state machine (or vice versa).

more

I am not mixing electric and diamondback, my debs are up to date (ros-electric-executive-smach: 1.0.2-s1313705059~natty, ros-electric-executive-smach-visualization 1.0.1-s1315010991~natty). I tried it on 64 and 32bit machines -> The Exception does still occur (a colleague of mine has the same issue)
( 2011-09-12 22:55:18 -0500 )edit