ntd_gpu autoware build from source failed
Hello,
I tried to install Autoware build from source https://github.com/Autoware-AI/autoware.ai/wiki/Source-Build , version 1.12.0 but failed.
System information:
Ubuntu 16.04
Memory 15.5 GiB
Processor Intel® Core™ i7-7700HQ CPU @ 2.80GHz × 8
Graphics GeForce GTX 1060/PCIe/SSE2
OS type 64-bit
Disk 83.0 GB
ROS version
mb@mb-Alienware-15-R3:~/autoware.ai$ env | grep ROS
ROSROOT=/opt/ros/kinetic/share/ros
ROSPACKAGEPATH=/opt/ros/kinetic/share
ROSMASTERURI=http://localhost:11311
ROSPYTHONVERSION=2
ROSVERSION=1
ROSLISPPACKAGEDIRECTORIES=
ROSDISTRO=kinetic
ROSETC_DIR=/opt/ros/kinetic/etc/ros
nvidia-smi
Wed Jan 27 18:28:07 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.27.04 Driver Version: 460.27.04 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 GeForce GTX 1060 Off | 00000000:01:00.0 On | N/A |
| N/A 43C P5 11W / N/A | 425MiB / 6073MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1086 G /usr/lib/xorg/Xorg 324MiB |
| 0 N/A N/A 2003 G compiz 90MiB |
| 0 N/A N/A 2567 G /usr/lib/firefox/firefox 1MiB |
| 0 N/A N/A 2686 G /usr/lib/firefox/firefox 1MiB |
| 0 N/A N/A 8084 G unity-control-center 1MiB |
| 0 N/A N/A 10189 G /usr/lib/firefox/firefox 1MiB |
+-----------------------------------------------------------------------------+
NVCC -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2015 NVIDIA Corporation
Built on TueAug1114:27:32CDT_2015
Cuda compilation tools, release 7.5, V7.5.17
Qt version:
Using Qt version 5.5.1 in /usr/lib/x86_64-linux-gnu
Issue: it generates ntd_gpu issue but every time it generates different packages processed (sometimes 99 packages not processed, sometimes 97, sometimes 9, etc,...)
mb@mb-Alienware-15-R3:~/autoware.ai$ AUTOWARECOMPILEWITHCUDA=1 colcon build --cmake-args -DCMAKEBUILDTYPE=Release Starting >>> autowarebuildflags Starting >>> autowaremsgs
Starting >>> vectormapmsgs Starting >>> autowareconfigmsgs
Starting >>> tabletsocketmsgs Starting >>> autowaresystemmsgs Starting >>> autowarecanmsgs Starting >>> gnss Finished <<< autowarebuildflags [17.3s]
Starting >>> sickldmrsdescription Finished <<< gnss [15.5s]
Starting >>> sickldmrsmsgs Finished <<< tabletsocketmsgs [15.9s]
Finished <<< autowaresystemmsgs [16.7s]
Starting >>> autowarehealthchecker Starting >>> ds4msgs
Finished <<< autowarecanmsgs [16.7s] Starting >>> custommsgs Finished <<< autowareconfigmsgs [17.7s]
Starting >>> ds4driver Finished <<< autowaremsgs [19.4s]
Starting >>> amathutilslib Finished <<< vectormapmsgs [20.2s]
Starting >>> vectormap Finished <<< sickldmrsdescription [5.35s]
Starting >>> astarsearch Finished <<< sickldmrsmsgs [6.33s]
Finished <<< ds4msgs [5.20s] Starting >>> sickldmrsdriver Starting >>> gazebocameradescription Finished <<< ds4driver [5.87s]
Starting >>> gazeboimudescription Finished <<< custommsgs [7.04s]
Starting >>> ndtcpu Finished <<< vectormap [6.62s]
Starting >>> vectormapserver Finished <<< gazebocameradescription [5.32s]
Starting >>> mapfile
Finished <<< gazeboimudescription [3.78s] Finished <<< amathutilslib [8.07s]
Starting >>> ndttku Starting >>> pclompregistration Finished <<< autowarehealthchecker [12.3s]
Starting >>> waypointfollower Finished <<< ndtcpu [7.82s]
Starting >>> fastvirtualscan Finished <<< pclompregistration [7.78s]
Starting >>> kittiboxpublisher Finished <<< ndttku [8.57s]
Starting >>> kittiplayer Finished <<< fastvirtualscan [7.50s]
Starting >>> ndtgpu Finished <<< sickldmrsdriver [17.3s] Starting >>> rosinterface Finished <<< astarsearch [18.5s]
Starting >>> runtimemanager Finished <<< mapfile [24.3s]
Starting >>> statemachinelib Finished <<< kittiboxpublisher [20.7s]
Starting >>> immukfpdatrack Finished <<< rosinterface [23.0s]
Finished <<< runtimemanager [23.1s]
Starting >>> libvectormap Starting >>> objectmap
Finished <<< statemachinelib [11.7s]
Starting >>> sickldmrstools Finished <<< waypointfollower [34.6s]
Finished <<< vectormapserver [36.8s]
Starting >>> oputility Starting >>> laneplanner
Finished <<< kittiplayer [52.1s]
Starting >>> waypointplanner --- stderr: ndtgpu
ptxas info : 0 bytes gmem /usr/include/string.h: In function ‘void* mempcpyinline(void, const void, sizet)’: /usr/include/string.h:652:42: error: ‘memcpy’ was not declared in this scope return (char *) memcpy (dest, _src, _n) + _n; ^ CMake Error at ndtgpugeneratedSymmetricEigenSolver.cu.o.cmake:266 (message): Error generating file /home/mb/autoware.ai/build/ndtgpu/CMakeFiles/ndtgpu.dir/src/./ndtgpugenerated_SymmetricEigenSolver.cu.o
make[2]: *** [CMakeFiles/ndtgpu.dir/src/ndtgpugeneratedSymmetricEigenSolver.cu.o] Error 1 make[1]: *** [CMakeFiles/ndt_gpu.dir/all] Error 2
make: *** [all] Error 2
Failed <<< ndtgpu [1min 8s, exited with code 2]
Aborted <<< sickldmrstools [43.4s]
Aborted <<< immukfpdatrack [50.8s]
Aborted <<< libvectormap [44.4s]
Aborted <<< laneplanner [43.2s]
Aborted <<< objectmap [44.5s]
Aborted <<< oputility [43.3s]
Aborted <<< waypointplanner [23.2s]
Summary: 32 packages finished [2min 1s]
1 package failed: ndtgpu
7 packages aborted: immukfpdatrack laneplanner libvectormap objectmap oputility sickldmrstools waypointplanner
1 package had stderr output: ndt_gpu
99 packages not processed
Asked by Mai Bui on 2021-01-27 19:54:49 UTC
Comments
This looks like a CUDA problem. What version of CUDA do you have installed?
Asked by Josh Whitley on 2021-01-30 16:41:01 UTC
It’s 11.2 version of CUDA
Asked by Mai Bui on 2021-01-30 16:43:19 UTC
According to nvcc -v command, it seems that your cuda is at 7.5. Based on requirements from https://gitlab.com/autowarefoundation/autoware.ai/autoware/-/wikis/Source-Build, for Ubuntu 16.04, version 9.0 is required.
For number of package differs at each compilation, this is not the issue, at least based on my experience. As multiple threads is used for compilation, there are some package that bound to be completed early (really depends on the system scheduler).
Asked by kurogane1031 on 2021-02-02 00:29:26 UTC