Kerberos is pretty cumbersome in practice. How about simple SSH tunnels using pre-shared key files (like Amazon EC2 does)? (That is the same as VPN, really, so you sort of answered your own question.) You would essentially be using the built-in AAA of Ubuntu's SSHd and extending it to all OS instances in the system.
EDIT:
To answer your question as to whether AAA is needed--leave aside the fact that Ubuntu already provides AAA. Here is a use case where security would be needed.
Let's say we are a company that has built a swarm of security robots (running ROS) that coordinates the patrolling of a mall to prevent terrorist bombings. The swarm includes bomb-sniffing ground robots, and the terrorists need to fool the robots into avoiding the areas where bombs have been placed.
To do so, the terrorists must connect to the swarm as a fake robot, running fake nodes that pretend to be the robots covering the areas where the bombs are placed and that report "all clear" to the other nodes.
In this case, all nodes would have to be secured well enough to guard against spoofing. SSH may not be secure enough for this purpose--once a hacker gets the SSH keys and establishes the tunnel, the traffic between ROS nodes itself is too trusting. It may be necessary, as you suggest, to integrate security into ROS itself.
(I am prepared for people to tear apart this use case and point out why security is or isn't an issue. I don't claim to be a security expert, and this is just a posed use case for the purpose of discussion.)
