How to automatically unpause a simulation when sim time is frozen?

edit

I'm attempting to synchronize two Gazebo simulations by listening to their clocks and pausing the faster simulation until the slower simulation catches up.

Each of the simulation are run on a different ROS core and different Gazebo master. The Multimaster FKIE package takes care of the communication between them. The /clock topic is ignored by the Multimaster FKIE topic, so that the /clock topic messages from both simulations don't get merged.

For each of the simulations there is a topic which publishes their clock, but with distinct name -- /slave_clock and /master_clock for the "slave" and "master" simulations respectively. Those topics are being shared across both ROS cores. I tried remapping the /clock topic but that didn't work. I'm not sure why.

The "slave" simulation is the slower one, so I run this synchronization node inside master ROS core. The Sincronization node reads the slave and master clocks, pauses master simulation when it gets ahead of the slave simulation and is supposed to unpause it when the slave simulation catches up. But that does not happen. I think the problem is that the synchronization node consumes simulation time, which is not running when the simulation is paused and therefor the node is not running and can't send the service call to unpause the simulation.

How can I resolve this issue?

EDIT

@gvdhoorn

The first thing I tried was to write a world gazebo_ros plugin that runs on the master world and reads the /slave_clock topic and on every slave_clock message desides whether to pause or unpause the simulation using the physics::pause_world(this->world, true) method.

The slave clock were taken from the message and the master clock obtained with ros::topic::waitForMessage<rosgraph_msgs::Clock>("/clock",ros::Duration(0.1)) method.

The simulation paused as expected, but it didn't unpause. I suspect for the same problem. The /clock topic was not active because sim time was freezed and no message was being published.

I don't know what approach you had in mind exactly. This is how I interpreted it.

EDIT 2

Now that I was forced to think about my plugin again, I have found a method in the Gazebo API that returns the world's simulation time physics::world::GetSimTime(). Using this method the plugin does not wait for the message that is not comming and the plugin is able to unpause the simulation.

This approach works. Thank you.