How to use the KITTI 3D object detection methods in our own camera-LiDAR setup, where we have only one calibration set?
I am working on real-time 3D object detection for an autonomous ground vehicle. The sensors that I use is a monocular camera and a VLP16 LiDAR. For extrinsic camera-LiDAR calibration and sensor fusion, I used the Autoware camera-LiDAR calibration tool. Now, I want to use the KITTI 3D object detection methods to obtain the 3D bounding boxes on an image. However, each image and its corresponding velodyne point cloud in the KITTI dataset have their own calibration file. In autoware, I am getting only a single extrinsic calibration file for the whole setup. Also, the Autoware calibration parameters are different from the calibration parameters of the KITTI dataset. Can anyone help me in converting the autoware calibration parameters to that of the KITTI dataset, or is there any other method to achieve my goal?
Autoware calibration file for the camera-LiDAR setup:
%YAML:1.0
CameraExtrinsicMat: !!opencv-matrix
rows: 4
cols: 4
dt: d
data: [ -9.9734380251035326e-01, 2.6761840324160763e-02,
6.7743217347964194e-02, 1.5696216374635696e-02,
-6.6967670310847577e-02, 2.8885241220820035e-02,
-9.9733694104487725e-01, 6.5152801573276520e-02,
-2.8647351139800312e-02, -9.9922442261089650e-01,
-2.7016338215861135e-02, -1.5130840241909027e-01, 0., 0., 0., 1. ]
CameraMat: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [ 2.2000230001872264e+03, 0., 1.0271964538622267e+03, 0.,
2.1884964850945166e+03, 6.4823423638043391e+02, 0., 0., 1. ]
DistCoeff: !!opencv-matrix
rows: 1
cols: 5
dt: d
data: [ -1.1189209437819703e-01, 1.0924203493113565e+00,
1.8551654586389964e-04, 3.3766563912686325e-03,
-2.8785099102921450e+00 ]
ImageSize: [ 1920, 1200 ]
ReprojectionError: 0
DistModel: plumb_bob
KITTI calibratio file for a pair of image and its corresponding velodyne point cloud:
P0: 7.215377000000e+02 0.000000000000e+00 6.095593000000e+02 0.000000000000e+00 0.000000000000e+00 7.215377000000e+02 1.728540000000e+02 0.000000000000e+00 0.000000000000e+00 0.000000000000e+00 1.000000000000e+00 0.000000000000e+00
P1: 7.215377000000e+02 0.000000000000e+00 6.095593000000e+02 -3.875744000000e+02 0.000000000000e+00 7.215377000000e+02 1.728540000000e+02 0.000000000000e+00 0.000000000000e+00 0.000000000000e+00 1.000000000000e+00 0.000000000000e+00
P2: 7.215377000000e+02 0.000000000000e+00 6.095593000000e+02 4.485728000000e+01 0.000000000000e+00 7.215377000000e+02 1.728540000000e+02 2.163791000000e-01 0.000000000000e+00 0.000000000000e+00 1.000000000000e+00 2.745884000000e-03
P3: 7.215377000000e+02 0.000000000000e+00 6.095593000000e+02 -3.395242000000e+02 0.000000000000e+00 7.215377000000e+02 1.728540000000e+02 2.199936000000e+00 0.000000000000e+00 0.000000000000e+00 1.000000000000e+00 2.729905000000e-03
R0_rect: 9.999239000000e-01 9.837760000000e-03 -7.445048000000e-03 -9.869795000000e-03 9.999421000000e-01 -4.278459000000e-03 7.402527000000e-03 4.351614000000e-03 9.999631000000e-01
Tr_velo_to_cam: 7.533745000000e-03 -9.999714000000e-01 -6.166020000000e-04 -4.069766000000e-03 1.480249000000e-02 7.280733000000e-04 -9.998902000000e-01 -7.631618000000e-02 9.998621000000e-01 7.523790000000e-03 1.480755000000e-02 -2.717806000000e-01
Tr_imu_to_velo: 9.999976000000e-01 7.553071000000e-04 -2.035826000000e-03 -8.086759000000e-01 -7.854027000000e-04 9.998898000000e-01 -1.482298000000e-02 3.195559000000e-01 2.024406000000e-03 1.482454000000e-02 9.998881000000e-01 -7.997231000000e-01
