[ndt_matching] gpu version died

edit

description

the ndt_matching node with pcl_generic method works fine,
but pcl_anh_gpu method always crashed when started

environment

gpu: nvidia gtx980m
gpu driver version: 410
cuda version: 10.0
cudnn version: 7.6.1.34
ros: melodic
autoware: 1.12.0 and master

steps to reproduce

1. in simulation tab, select bag file (demo data provided by autoware), press start and pause
2. in setup tab, press tf and vehicle model
3. in map tab
    select and press point cloud (demo data provided by autoware)
    select and press vector map (demo data provided by autoware)
    select and press tf (demo data provided by autoware)
4. in simulation tab, press pause again to resume, 
    in rviz, verify that the pcd map and vector map are loaded correctly
5. in sensing tab, click voxel_grid_filter with default parameters
6. in computing tab, click nmea2tfpose with default parameters
    select pcl_anh_gpu method and click ndt_matching
7. in simulation tab, press pause to resume
8. then ndt_matching died with error message as below

error messages

Error: out of memory /home/leon/autoware.ai/src/autoware/core_perception/ndt_gpu/src/VoxelGrid.cu 181 terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::lock_error> >' what():  boost: mutex lock failed in pthread_mutex_lock: Invalid argument [ndt_matching-1] process has died [pid 8436, exit code -6, cmd /home/leon/autoware.ai/install/lidar_localizer/lib/lidar_localizer/ndt_matching __name:=ndt_matching __log:=/home/leon/.ros/log/a41e9918-b072-11e9-96f6-9cb6d01138d9/ndt_matching-1.log]. log file: /home/leon/.ros/log/a41e9918-b072-11e9-96f6-9cb6d01138d9/ndt_matching-1*.log

contents of ndt_matching-1-stdout.log

Log file: method_type: 2 use_gnss: 1 queue_size: 1 offset: linear get_height: 0 use_local_transform: 0 use_odom: 0 use_imu: 0 imu_upside_down: 0 imu_topic: /imu_raw localizer: velodyne (tf_x,tf_y,tf_z,tf_roll,tf_pitch,tf_yaw): (1.2, 0, 2, 0, 0, 0) Update points_map.