Revision history [back]

The answer for this question covers a subjectively large scope of study. There are various modules that we may have in autonomous vehicle. For example, if we are talking about camera perception engine, then the subsequent question is what end result(s) that we want to evaluate: is it depth map, is it object detection, is it lane detection as segmentation, is it lane detection as LDWS, etc.

However, if the question is focused only on (for instance) localization from Autoware.Auto, you may need to consider few aspects to focus in:

• metrics to evaluate from general perspective (e.g. runtime, complexity, iterations),
• metrics to evaluate from algorithm perspective (e.g. translation accuracy, rotational accuracy)
• metrics to evaluate from embedded system implementation perspective (e.g. binary size, memory requirement)

If you are looking to focus on general perspective, you can look into: https://www.hanselman.com/blog/content/binary/NDepend%20metrics%20placemats%201.1.pdf

If you are looking to focus on algorithm perspective, you can search relevant journal publications or any open-source qualitative evaluation depending on which one you want to evaluate (I don't think there is all-in-one source for this).