ntd_gpu autoware build from source failed

asked 2021-01-27 18:54:49 -0500

Mai Bui gravatar image

updated 2021-01-28 21:56:55 -0500

Hello,

I tried to install Autoware build from source https://github.com/Autoware-AI/autowa... , version 1.12.0 but failed.
System information: Ubuntu 16.04
Memory 15.5 GiB
Processor Intel® Core™ i7-7700HQ CPU @ 2.80GHz × 8
Graphics GeForce GTX 1060/PCIe/SSE2
OS type 64-bit
Disk 83.0 GB

ROS version
mb@mb-Alienware-15-R3:~/autoware.ai$ env | grep ROS
ROS_ROOT=/opt/ros/kinetic/share/ros
ROS_PACKAGE_PATH=/opt/ros/kinetic/share
ROS_MASTER_URI=http://localhost:11311
ROS_PYTHON_VERSION=2
ROS_VERSION=1
ROSLISP_PACKAGE_DIRECTORIES=
ROS_DISTRO=kinetic
ROS_ETC_DIR=/opt/ros/kinetic/etc/ros

nvidia-smi

Wed Jan 27 18:28:07 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.27.04 Driver Version: 460.27.04 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 GeForce GTX 1060 Off | 00000000:01:00.0 On | N/A |
| N/A 43C P5 11W / N/A | 425MiB / 6073MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1086 G /usr/lib/xorg/Xorg 324MiB |
| 0 N/A N/A 2003 G compiz 90MiB |
| 0 N/A N/A 2567 G /usr/lib/firefox/firefox 1MiB |
| 0 N/A N/A 2686 G /usr/lib/firefox/firefox 1MiB |
| 0 N/A N/A 8084 G unity-control-center 1MiB |
| 0 N/A N/A 10189 G /usr/lib/firefox/firefox 1MiB |
+-----------------------------------------------------------------------------+

NVCC -V

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2015 NVIDIA Corporation
Built on Tue_Aug_11_14:27:32_CDT_2015
Cuda compilation tools, release 7.5, V7.5.17

Qt version:
Using Qt version 5.5.1 in /usr/lib/x86_64-linux-gnu

Issue: it generates ntd_gpu issue but every time it generates different packages processed (sometimes 99 packages not processed, sometimes 97, sometimes 9, etc,...)

mb@mb-Alienware-15-R3:~/autoware.ai$ AUTOWARE_COMPILE_WITH_CUDA=1 colcon build --cmake-args -DCMAKE_BUILD_TYPE=Release Starting >>> autoware_build_flags Starting >>> autoware_msgs
Starting >>> vector_map_msgs Starting >>> autoware_config_msgs
Starting >>> tablet_socket_msgs Starting >>> autoware_system_msgs Starting >>> autoware_can_msgs Starting >>> gnss Finished <<< autoware_build_flags [17.3s]
Starting >>> sick_ldmrs_description Finished <<< gnss [15.5s]
Starting >>> sick_ldmrs_msgs Finished <<< tablet_socket_msgs [15.9s]
Finished <<< autoware_system_msgs [16.7s]
Starting >>> autoware_health_checker Starting >>> ds4_msgs
Finished <<< autoware_can_msgs [16.7s] Starting >>> custom_msgs Finished <<< autoware_config_msgs [17.7s]
Starting >>> ds4_driver Finished <<< autoware_msgs [19.4s]
Starting >>> amathutils_lib Finished <<< vector_map_msgs [20.2s]
Starting >>> vector_map Finished <<< sick_ldmrs_description [5.35s]
Starting >>> astar_search Finished <<< sick_ldmrs_msgs [6.33s]
Finished <<< ds4_msgs [5.20s] Starting >>> sick_ldmrs_driver Starting >>> gazebo_camera_description Finished <<< ds4_driver [5.87s]
Starting >>> gazebo_imu_description Finished <<< custom_msgs [7.04s]
Starting >>> ndt_cpu Finished <<< vector_map [6.62s]
Starting >>> vector_map_server Finished <<< gazebo_camera_description [5.32s]
Starting >>> map_file
Finished <<< gazebo_imu_description [3.78s] Finished <<< amathutils_lib [8.07s]
Starting >>> ndt_tku Starting >>> pcl_omp_registration Finished <<< autoware_health_checker [12.3s]
Starting >>> waypoint_follower Finished <<< ndt_cpu [7.82s]
Starting >>> fastvirtualscan Finished <<< pcl_omp_registration [7.78s]
Starting >>> kitti_box_publisher Finished <<< ndt_tku [8.57s]
Starting >>> kitti_player Finished <<< ...

(more)
edit retag flag offensive close merge delete

Comments

This looks like a CUDA problem. What version of CUDA do you have installed?

Josh Whitley gravatar image Josh Whitley  ( 2021-01-30 15:41:01 -0500 )edit

It’s 11.2 version of CUDA

Mai Bui gravatar image Mai Bui  ( 2021-01-30 15:43:19 -0500 )edit

According to nvcc -v command, it seems that your cuda is at 7.5. Based on requirements from https://gitlab.com/autowarefoundation..., for Ubuntu 16.04, version 9.0 is required.

For number of package differs at each compilation, this is not the issue, at least based on my experience. As multiple threads is used for compilation, there are some package that bound to be completed early (really depends on the system scheduler).

kurogane1031 gravatar image kurogane1031  ( 2021-02-01 23:29:26 -0500 )edit