It depends on what planning algorithms you want to compare, and what you want the desired behavior of the robot to be. I'm not sure of any gazebo features for directly measuring metrics though.

Here are some ideas to help. Some standard path-planning metrics that come to mind:

• Time-to-replan: How long does your algorithm take to create a new path? Easy to implement, just get time before and after planning, and take difference. Could be interesting as more obstacles are added.
  • Note: This metric is especially useful if only local obstacles are visible to the planning algorithm, so re-planning around newly sensed obstacles is required. This metric would also make sense when considering dynamic obstacles.
• Success rate: How often does the path planner succeed at getting the robot to the goal. Easy to implement, just run the planning algorithm many times with varying initial and final conditions, and report the percent of time the robot was able to get to the goal.
  • Caveat: this metric mainly makes sense for "local" planning, in which the robot can only see local obstacles. Obviously, success rate would be 100% if there exists a feasible path, and the algorithm has full knowledge of the existing obstacles (or else it is a very bad algorithm).
• Length of path: How short is the planned path? Harder to implement, but not by much.
  • Implementation option: If we assume we get position info from the robot at a high frequency, a pretty good approximation of path length is adding up the distances between consecutive position data points (see 330377).

There are many more metrics, but it just depends on your use case! Hope this helps point you in the right direction.