hey guys , I am working on a pose estimation problem for objects from their 3D pointcloud in the scene using traditional techniques (like PCA / OBB ) within the PCL library. I have got this thing working but sometimes I am having erroneous absolute orientation values for objects. Eg: In this image below you can see pose visualization for two cones (left and right , the camera pose is in the center).

Here is the image visualization link for the issue.

The 3D position of the cones are correct but The absolute orientation of the cones is incorrect. Both the cones should have the same absolute orientation values and they should be parallel to the camera pose.

The Viz is created using this code block: pcl::MomentOfInertiaEstimation pcl::PointXYZRGB feature_extractor;

featureextractor.setInputCloud (cloudptr);

feature_extractor.compute();

std::vector momentofinertia;

std::vector eccentricity;

featureextractor.getMomentOfInertia (momentof_inertia);

feature_extractor.getEccentricity (eccentricity);

featureextractor.getAABB (minpointAABB, maxpoint_AABB);

featureextractor.getOBB (minpointOBB, maxpointOBB, positionOBB, rotationalmatrixOBB);

Eigen::Quaternionf quat (rotationalmatrixOBB);

The pointcloud input is in odom frame. Could this be a bad transform problem on the rotation matrix calculated ? I think I am missing some information on how to go on about doing this. Is what I am saying even feasible ? Is it possible to constrain the absolute pose of the objects some how ? Can anyone point me to any resources that help me regarding this problem ?