We ran actually into the same problem of needing deterministic data processing when receiving data from rosbags as @janismac. @asmodehn nicely summarized the points that are important when implementing a node with deterministic behaviour.
Considering the ROS communication part, we implemented a very simple handshake mechanism in rosbag playback to prevent message loss: We modified rosbag to keep track of the filling state of each registered node. Whenever rosbag sends out a message the queue filling counter for the corresponding node is decreased. When the corresponding subscriber actually processes a received message it signals a free slot in the input queue by a service call to rosbag. If one of the monitored queues is full, rosbag automatically pauses playback until the corresponding node signals a free queue slot again. The mechanism is only active if use_sim_time is set to true so it does not influence message processing when using “real” sensor data.
The system works fine for linear data flows, meaning that one message (block) sent from rosbag to node A maps exactly to one message (block) which is passed from node A to node B and so on. Message BLOCK means a couple of messages which belong together (for example a camera image and a corresponding camera_info message). With the mechanism you can set playback speed to lets say -r 100 and the processing will run at the speed of the slowest node in the processing chain.
We uploaded the modified rosbag and the rc_bagthrottler module to github:
https://github.com/roboception
At the current state of development, the documentation is a little bit rudimentary and there is still some “zombie” code. We are going to extend it and clean up the code very soon. Until now, it was not necessary for us to extend the mechanism for a further handshake directly between nodes but we would be happy about discussions on the subject.