ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question

Autoware: crosscompile with CUDA support

asked 2020-01-21 07:32:55 -0500

Fernnando33 gravatar image

Hi everyone! I was working with Autoware in my jetson tx2 board and everything is fine when I compile natively. Now, I want to be able to crosscompile Autoware within my desktop computer. I followed all the steps that I found in Autoware gitlab and forums and I was able to crosscompile Autoware 1.12 with the docker image. However, I could see that CUDA is not enabled and I really need it.

Is there any way to add CUDA support to docker crosscompilation? I'm pretty new with docker so it's hard for me to detect wheter some dependency is missing or if I have to add some environment variable. When I add AUTOWARE_COMPILE_WITH_CUDA=1 to crosscompile script I have the following error:

~/$ ./colcon_release_cross kinetic generic-aarch64
Starting >>> autoware_build_flags
Starting >>> autoware_msgs
Starting >>> vector_map_msgs
Starting >>> autoware_config_msgs
Starting >>> tablet_socket_msgs
Starting >>> autoware_system_msgs
Starting >>> autoware_can_msgs
Starting >>> gnss
Finished <<< autoware_build_flags [16.3s]                                                                                                                                                                        
Finished <<< gnss [16.4s]                                                                                                                                                        
Starting >>> ds4_msgs
Starting >>> custom_msgs
Finished <<< autoware_can_msgs [24.4s]                                                                                                                                                                      
Finished <<< autoware_system_msgs [24.4s]
Finished <<< tablet_socket_msgs [24.4s]
Starting >>> ds4_driver
Starting >>> autoware_health_checker
Starting >>> gazebo_camera_description
Finished <<< autoware_config_msgs [31.0s]                                                                                                                                                                          
Starting >>> gazebo_imu_description
Finished <<< autoware_msgs [31.0s]                                                                                                                                                      
Starting >>> amathutils_lib
Finished <<< ds4_msgs [16.9s]                                                                                                                                                                                        
Starting >>> astar_search
Finished <<< vector_map_msgs [35.8s]                                                                                                                                                                         
Starting >>> vector_map
Finished <<< gazebo_camera_description [11.4s]
Finished <<< ds4_driver [11.4s]
Starting >>> ndt_cpu
Starting >>> ndt_tku
Finished <<< gazebo_imu_description [16.4s]                                                                                                                                                                        
Finished <<< custom_msgs [30.8s]
Starting >>> pcl_omp_registration
Starting >>> fastvirtualscan
Finished <<< amathutils_lib [45.3s]                                                                                                                                                                                  
Starting >>> kitti_box_publisher
Finished <<< ndt_tku [47.3s]                                                                                                                                                                                  
Finished <<< vector_map [47.3s]
Starting >>> vector_map_server
Starting >>> map_file
--- stderr: astar_search                                                                                                                                                                                   
/home/autoware/Autoware/src/autoware/core_planning/astar_search/test/src/test_class.cpp: In constructor 'TestClass::TestClass()':
/home/autoware/Autoware/src/autoware/core_planning/astar_search/test/src/test_class.cpp:54:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int idx = 0; idx <*; ++idx)
/home/autoware/Autoware/src/autoware/core_planning/astar_search/test/src/test_class.cpp:60:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int row = 0; row <; ++row)
/home/autoware/Autoware/src/autoware/core_planning/astar_search/test/src/test_class.cpp:62:27: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     for (int col = 0; col <; ++col)
/home/autoware/Autoware/src/autoware/core_planning/astar_search/test/src/test_astar_search.cpp: In member function 'virtual void TestSuite_checkSetPath_Test::TestBody()':
/home/autoware/Autoware/src/autoware/core_planning/astar_search/test/src/test_astar_search.cpp:143:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int row = 0; row <; ++row)
/home/autoware/Autoware/src/autoware/core_planning/astar_search/test/src/test_astar_search.cpp:145:27: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     for (int col = 0; col <; ++col)
Finished <<< astar_search [57.9s]
--- stderr: ndt_cpu
** WARNING ** io features related to openni2 will be disabled
** WARNING ...
edit retag flag offensive close merge delete


I don't see any cuda tags for autoware arm64v8 repo. Although the (implicit) amd64 repo seems to have cuda tags. It seems that the amd64 images install cuda by inheriting FROM nvidia's opengl parent image. Looks like there may be beta support for arm64v8 for nvidia-docker plugin though, so see if you could swap the parent image appropriately.

ruffsl gravatar image ruffsl  ( 2020-01-21 13:01:25 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted

answered 2020-01-26 21:18:45 -0500

Josh Whitley gravatar image

The current Docker containers do not support CUDA for cross-compilation. Doing so would require a pretty significant effort as the cross-compilation images are somewhat complex. We use an alpine image to bootstrap the creation of a "native" (using binfmt) arm64 image where all of the dependencies are installed. We then build an amd64 image on Ubuntu and copy all necessary environment from the intermediate arm64 image to /sysroot/generic-aarch64 in the amd64 environment. So, basically, you would have to do the following:

  1. Add another Dockerfile which uses the same process as the existing cross-compilation setup.
  2. Have that new Dockerfile install CUDA in the intermediate arm64 environment.
  3. Copy all of the CUDA components from the arm64 environment into /sysroot/generic-aarch64/ in the amd64 environment.
  4. Modify any scripts or CMake files which look for CUDA (such as the FindCuda.cmake file you mention above) to look in /sysroot/generic-aarch64/usr/local/cuda instead of /usr/local/cuda.

Unfortunately, the community hasn't shown enough interest thus far in a cross-compilation-supporting-CUDA image for anyone to make the effort. However, if you're willing to give it a shot, I'm happy to put in a bit of time to help.

edit flag offensive delete link more

Question Tools

1 follower


Asked: 2020-01-21 07:32:55 -0500

Seen: 460 times

Last updated: Jan 26 '20