region_tlr_ssd Segmentation fault error

edit

Hello everyone,

When I run the region_tlr_ssd node in Autoware 1.13, for few seconds(or approx. 1 minute) it is giving me Segmentation Fault error. Here is my nvidia-smi output when I run the node:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 435.21       Driver Version: 435.21       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce RTX 207...  Off  | 00000000:01:00.0  On |                  N/A |
| N/A   63C    P0    42W /  N/A |   6514MiB /  7982MiB |     56%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1944      G   /usr/lib/xorg/Xorg                           462MiB |
|    0      2112      G   /usr/bin/gnome-shell                         284MiB |
|    0      3033      G   ...AAAAAAAAAAAAAAgAAAAAAAAA --shared-files   155MiB |
|    0      7780      G   ...uest-channel-token=12843037902862002010   157MiB |
|    0     12644    C+G   /opt/ros/melodic/lib/rviz/rviz               156MiB |
|    0     13322      C   .../trafficlight_recognizer/region_tlr_ssd  5286MiB |
+-----------------------------------------------------------------------------+

System: - Graphics Card: RTX 2070

• CUDA: 10.0
• Driver: 435
• OpenCV Version from python3:

>>> import cv2
>>> cv2.__version__
'4.1.2'

-OpenCV Version from pkg-config:

$ pkg-config --modversion opencv
3.2.0
$ cat $(locate OpenCVConfig.cmake) | grep "SET(OpenCV_VERSION "
SET(OpenCV_VERSION 4.1.2)
SET(OpenCV_VERSION 4.1.2)
SET(OpenCV_VERSION @OPENCV_VERSION_PLAIN@)
SET(OpenCV_VERSION 3.2.0)

• CuDNN: 7.6.5

$ cat /usr/include/cudnn.h | grep CUDNN_MAJOR -A 2
#define CUDNN_MAJOR 7
#define CUDNN_MINOR 6
#define CUDNN_PATCHLEVEL 5
--
#define CUDNN_VERSION (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL)

I am using https://github.com/weiliu89/caffe/tre... as repo. Here is my Makefile.config,

run all tests with no problem such as:

make test -j8
make runtest -j8

Also, I specified the ssdcaffe path in CMakeLists.txt of TrafficLightRecognizer module and put the label_map_file path in deploy.prototxt (I used every model in the repo, none of them changed anything,)

I tried to give the real image sizes in deploy.prototxt but it only delayed the Segmentation Fault:

name: "VGG_VOC0712Plus_SSD_512x512_ft_deploy"
input: "data"
input_shape {
  dim: 1
  dim: 3
  dim: 1920
  dim: 1440
}...

Here is my OUTPUT of running the region_tlr_ssd.