If so do you just set the shebang to use that specific environment?

Yes, for instance. In the end roslaunch just starts scripts or binaries (anything executable really). You'll have to carefully manage the environment (such as PYTHONPATH), but as you seem to be mixing different versions of things anyway, you'll have to do that, no matter what.

In a virtualenv (or whatever the flavour of isolation tool is) the interpreter specified in the shebang will be local to that environment. To be able to run a script "in" that environment, all you need to do is use its interpreter. As long as you take care of special environment variables (ie: the ones the interpreter and related tools/libraries/binaries care about), things should just work.

That's also how catkin_virtualenv deals with this.

I'm not even sure if this would be useful since you couldn't publish or subscribe to any topics with the Python 3.7 node.

Why not? All you need is the rospy library (and messages) to be present on the PYTHONPATH.

I've been using rospypi/simple for that (just because it's stand-alone, small, lightweight, etc).

What do the rest of you do if you need to use a Python 3 specific package with ROS?

In ROS Melodic you mean? That's always going to be somewhat more complicated, as it doesn't support Python 3, so none of the base dependencies will be present on the PYTHONPATH after you source your setup.bash. ROS Noetic only supports Python 3, so you should not have that problem any more. Alternatively: build Melodic from source if you really want a full ROS environment with Python 3 support through-and-through.

I don't think it's currently possible since ROS doesn't like it when you mess with it defaulting to the system wide distro of Python 2.7.

I would say this is not ROS specific: Debian/Ubuntu packages (but the same goes for many more system package managers) just get "tied" to specific versions of their dependencies. There are ways around that, but in the end, those may not be any better than releasing different versions of the packages themselves -- which is what will happen, with ROS Noetic.

Before someone points me to a link of how to use ROS with Python 3, please hold off. I've seen those. I got'em bookmarked, but what if I need to use Python 2.7 and 3.8?

This also doesn't seem ROS specific, unless you are strictly thinking about using ROS tools from the "Python 2 side" to start things from the "Python 3 side". But then I would still expect what I wrote above to hold (ie: use the right shebang).

Mixing different Python versions inside the same virtualenv/pyvenv/whatever is also not possible (without hacks), and the same restrictions and caveats apply here.

if it's currently possible to roslaunch two different python version nodes (one that uses Python 2.7 and one that uses Python 3.7 for example)?

With the appropriate care, I would say yes (I've done it myself), but seeing as it's messy in any case, I cannot guarantee it will work for whatever framework/library/infrastructure you are trying to use.

I would take a look at catkin_virtualenv as well. It's definitely not perfect and comes with its own limitations, but there are Python 2 and Python 3 versions of it, so perhaps it could help you overcome whatever is limiting you right now.

Comments

Very thorough answer @gvdhoorn. Thank you. This line of your answer really stuck out to me: "Python 2 side" to start things from the "Python 3 side". I think is what I was getting at in my convoluted question. Here is how I saw this gong.

1. Start roscore inside the base system environment, configured for Python 2.7.
2. Start a ros node that uses python 2.7 an image_publisher for example
3. Start another ros node that uses python 3.7 an image_subscriber and processor for example that uses opencv (which requires a python 3 dependency, which is why the node needs to run off Python 3.7).

Now would you need to be within a conda, or pin, or catkin_virtualenv in that third terminal in order to start that node without errors, or could you just specify an interpreter in the shebang? The way I understand is that starting a Python 3.7 virtual environment is the same as writing a shebang in a script that uses the same python distro that the virtual environment uses (from conda or pyenv)

Asked by rational_galt on 2020-05-23 14:23:37 UTC

However, I just realized that I would not be able to execute the command rosrun in that third terminal if I was in a Python 3.7 virtual environment. So maybe that's the workaround? Just put the python 3.7 interpreter (that you can get installed on your system with pyenv) in the shebang for the ROS node. But don't turn that terminal into a conda or pipenv or other virtual environment because then ROS won't work if you installed it the normal way and not a custom install from source. Even if you did install ROS from source and set it up to use Python3 instead of Python2, you wouldn't be able to start any nodes that require python2.7, such as the image converter in this example. Hope that all makes sense.

Asked by rational_galt on 2020-05-23 14:28:05 UTC

Just put the python 3.7 interpreter (that you can get installed on your system with pyenv) in the shebang for the ROS node

exactly.

That would be the first thing to do and then see whether things work (note that if your packages contain any catkin_install_python(..) lines it may cause the shebang to be rewritten to whatever interpreter Catkin uses at that point, so test this with a devel space).

It's going to be hacky in any case, but as I wrote, that's not specific to ROS.

Compare what you're trying to do with mixing and matching bits and pieces from different venvs or Conda environments. That's also not a supported use case (afaik). This is the exact same thing. It just happens that "the ROS Python libraries" are involved.

Asked by gvdhoorn on 2020-05-24 04:23:54 UTC

However, I just realized that I would not be able to execute the command rosrun in that third terminal if I was in a Python 3.7 virtual environment

which is not really a problem, as rosrun is just a thin wrapper script which does a rospack find <package name> for you and then invokes your .py, while passing some standard arguments to it (notable __name and __log).

You don't need rosrun to start ROS nodes. It's just convenience.

Asked by gvdhoorn on 2020-05-24 04:39:06 UTC