Ask Your Question
2

Python Multiprocessing to publish to multiple topics

asked 2014-01-19 23:21:42 -0500

Mandar gravatar image

updated 2014-01-28 17:19:05 -0500

ngrennan gravatar image

I currently have a setup where I have about 10 shell scripts that listen to 10 different KNX addresses (using the shell command grouplisten) and publish their state to a ROS topic. These scripts can run in the background, so it's quite convenient to 'listen' to the state of these devices.

I tried to replace these 10 scripts with a single python program that basically does this:

  1. Declare a subprocess.Popen for each device, to listen to its knx device.

    self.listen_stat = subprocess.Popen(self.cmds,stdout=subprocess.PIPE,preexec_fn=os.setsid)
    
  2. Declare the node (in __main__) , and topic for each device ( in device.__init__()).

    class DeviceProp:
    def __init__(self):
      ...
      self.pub = rospy.Publisher(self.name,Int32,latch=True)
    
  3. Call a talker function like so:
try:
  for device in device_props:
       p = multiprocessing.Process(target=device.device_talker)
       procs.append(p)
       p.start()
except (KeyboardInterrupt,rospy.ROSInterruptException,IOError) as e:
    for p in procs:
       p.join()

(The device_talker basically just publishes new data to device.pub)

What happens is that the logs suggest data is being published, but I don't see anything on when I subscribe to those topics or if I do rostopic echo. If I run rostopic pub from a new shell, I get the data on the topic as expected. I've also added a rospy.sleep(1) at multiple places (in loops too) so that even if I lose the first few messages, I _should_ get the later ones. But I don't.

Any suggestions on what I should do? (In case you are wondering why - I want to replace 10 scripts with 1 for easier code management. Also, Since I can store all address and device props in a yaml file, it will be trivial to add, or edit the configuration file).

edit retag flag offensive close merge delete

Comments

I realize that I may be using the multiprocessing module wrong to share variables (the publisher for instance). It would be helpful if someone could suggest a way to fix that.

Mandar gravatar imageMandar ( 2014-01-20 01:52:46 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
2

answered 2014-01-20 23:18:27 -0500

Mandar gravatar image

I managed to solve this problem (Note to self: Read the python docs). Turns out that a new process thread creates a copy of all the data from the parent, and I had not taken care of this properly.

The way I solved it was thus: I called the Popen (listen_stat in this case) from the device method get_status:

 def get_status(self):
    varnum = 0  # for testing only
    while not rospy.is_shutdown():
       self.response_line = self.listen_stat.stdout.readline().split()
       self.new_detect.value = int(self.response_line[varnum])

The important detail was declaring the variable new_detect as a multiprocessing.Value() and then calling this function instead for the multiprocess.

stat_procs = []
for device in device_props:
    sp = multiprocessing.Process(target=device.get_status)
    stat_procs.append(sp)
    sp.start()

The actual publisher then was just called in a normal loop:

try:
    while not rospy.is_shutdown():
        for device in device_props:
            device.device_talker()
except (KeyboardInterrupt,rospy.ROSInterruptException,IOError) as e:
    for p in stat_procs:
       p.join()

Hope this helps the others.

edit flag offensive delete link more

Comments

Hi, I am trying to do the same thing. However, I want to create multiple publishers in a single ros node. But when I call the publish method inside a while loop, only the first called publisher is invoked.

Could you please suggest suitable multi threading for this purpose ?

amarbanerjee23 gravatar imageamarbanerjee23 ( 2017-06-22 01:57:50 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2014-01-19 23:21:42 -0500

Seen: 2,093 times

Last updated: Jan 20 '14