Install on shared HPC cluster as normal user without using OS repos
Hi,
One of our users has asked for us to install a package (FlightGoggles) that depends on ROS. However, I am struggling to get ROS working.
The base OS image for our system is a _read-only_, minimal CentOS 6 install. All centrally-installed software packages that we've installed for users live in our /apps directory, where they're separate and versioned (for example -- the Python installation that I've been using is at /apps/python3/3.6.7). We use environment modules to manage access to these -- e.g. module load python3/3.6.7 makes the needed changes to the environment such that that Python install is usable.
I've so far managed to get rosdep to install in my own directory. I needed to patch one file to change the hard-coded(!) /etc/ros to something that is writable, and then rosdep init and rosdep update worked properly. However, I cannot get the Catkin workspace to build.
I'm following the instructions at http://wiki.ros.org/kinetic/Installat..., and can download the source packages, but attempting to install them fails:
$ rosinstall_generator desktop --rosdistro kinetic --deps --wet-only --tar > kinetic-desktop-wet.rosinstall
<success>
$ wstool init -j8 src kinetic-desktop-wet.rosinstall
<success>
$ rosdep install --from-paths src --ignore-src --rosdistro kinetic -y
rosdep detected OS: [centos] aliasing it to: [rhel]
ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
rospack: No definition of [python-coverage] for OS [rhel]
rqt_logger_level: No definition of [python-rospkg] for OS [rhel]
roslz4: No definition of [lz4] for OS [rhel]
rqt_robot_monitor: No definition of [python-rospkg] for OS [rhel]
rqt_bag: No definition of [python-rospkg] for OS [rhel]
webkit_dependency: No definition of [python-qt5-bindings-webkit] for OS [rhel]
roscreate: No definition of [python-rospkg] for OS [rhel]
roslaunch: No definition of [python-rospkg] for OS [rhel]
qt_gui_py_common: No definition of [python-rospkg] for OS [rhel]
actionlib: No definition of [python-wxtools] for OS [rhel]
rqt_publisher: No definition of [python-rospkg] for OS [rhel]
python_qt_binding: No definition of [python-qt5-bindings] for OS [rhel]
catkin: No definition of [python-nose] for OS [rhel]
urdf: No definition of [liburdfdom-headers-dev] for OS [rhel]
qt_gui: No definition of [tango-icon-theme] for OS [rhel]
collada_parser: No definition of [collada-dom] for OS [rhel]
rqt_pose_view: No definition of [python-rospkg] for OS [rhel]
rqt_top: No definition of [python-psutil] for OS [rhel]
rosclean: No definition of [python-rospkg] for OS [rhel]
rqt_console: No definition of [python-rospkg] for OS [rhel]
geometric_shapes: No definition of [libqhull] for OS [rhel]
rosgraph: No definition of [python-rospkg] for OS [rhel]
rqt_robot_steering: No definition of [python-rospkg] for OS [rhel]
rqt_runtime_monitor: No definition of [python-rospkg] for OS [rhel]
roslisp: No definition of [sbcl] for OS [rhel]
resource_retriever: No definition of [python-rospkg] for OS [rhel]
rqt_web: No definition of [python-rospkg] for OS [rhel]
rviz: No definition of [liburdfdom-headers-dev] for OS [rhel]
gl_dependency: No definition of [python-qt5-bindings-gl] for OS [rhel]
kdl_parser: No definition of [liburdfdom-headers-dev] for OS [rhel]
rqt_dep: No definition of [python-rospkg] for OS [rhel]
rqt_topic: No definition of [python-rospkg] for OS [rhel]
rqt_service_caller: No definition of [python-rospkg] for OS [rhel]
robot_state_publisher: No definition of [liburdfdom-headers-dev ...
This is not an answer, hence a comment, but I believe this is exactly what Singularity was created for. It's a container technology stack that is specifically targeting HPC and cluster environments with security setups that are incompatible with how those kinds of environments are typically setup.
I would definitely recommend you look at that.
You're student would create an image on his own machine, upload it (in some way) to the cluster and then use it for his experiments without ever needing any write access to anything or super user permissions.
It's Docker compatible (as in: can consume Docker images) and the newer versions support things like access to compute accelerators, GPUs and other resources a cluster might have.
(and I've linked you to the old documentation on purpose as I find it slightly more beginner friendly. You'll want to go to the main ...(more)
Thanks. Yes, Singularity was my next go to. I was just hoping there was a way to do a native install as well; we put a lot of effort into optimising the packages in the central store, so it seems a waste to not make use of them.
Which packages?
ROS is a CBSE framework. There are literally thousands of packages. Would it still scale to treat that as you do regular, stand-alone pieces of software?
From a maintenance perspective I would not want to have to deal with all of that. Using containers seems like an efficient way to scale this without losing control over your infrastructure.
I don't need ROS per se on its own, I just need it to install the real program the user is after (FlightGoggles). I couldn't find a normal autoconf or CMake way of building that, only a rosinstall script.
Thing is: I don't believe you can treat ROS applications the same as you'd do with other, more monolithic software.
FlightGoggles itself already consists of 6 packages. Those pkgs directly depend on 3 others, and all of that combined depends on 49 other pkgs (
rospack depends).All of those packages are essentially stand-alone programs or libraries, all of which come with
CMakeLists.txtorMakefiles that you could, theoretically,mkdir build && cd build && cmake ..and/ormake && ...I don't know of anyone doing that manually though, as it doesn't scale with typical ROS applications.
Would you still refer to those N packages as "the real program"? You cannot install just FlightGoggles. It needs all of those other packages as well.
Packaging all of that in a container -- which contains a ROS workspace and has been setup following the build setup and deployment ...(more)
Hmm, yes, that's sounding like a better idea at this stage. Thanks!