I have used Unity ROS finally to have this solved which supports Universal Platform Support; which implies it can be used for VR devices, and similar application.
I had to make some changes to the data model that is sent over the rosbridge. Kindly follow Issue #11 for further details.