Managing large video streams (4K ...)
Hi, We want to equip our car mounted system with recording capability for high-quality video streams, possibly several Full HD and maybe 4K cameras. We need frames from those cameras to be recorded with a timestamps synchronized with other sensors (GPS,...) and to be stored at reasonable framerate (15-30 FPS). What is generally a good approach for implementing this?
Currently we have 4 Full HD webcams in the system connected to a laptop through a single USB 2.0 port and we already face serious problems. The USB capacity prevents us to set the full resolution (ROS complains on startup of the nodes) so we have to run at 720p and 15 fps. Even with this, we only record around 7 fps on our Core i7 laptop. I suspect that the on-the-fly compression to jpeg (frames are stored compressed) can be the problem. I do not see this scaling very well. Do we need multiple USB controllers, multiple PCs, LAN based cameras (e.g. PointGrey)? Or is recording frames as JPEG images instead of a video codec (e.g. H264) even a good practice? It seems very wasteful to me. Is a more reasonable approach to record each stream as a separate video using an external SW (not ROS) and just note the time of the first frame? Would timing for the following frames be reliable? We can tolerate error of 500 ms over recording duration of 1 hour.
Thanks!
I have no real experience with this, but if you're not processing your video in real-time / while capturing it, I would try not to use ROS for the capture. As you already wrote, there are other ways to capture video streams, which are probably more efficient than storing individual raw / jpeg shots in a directory / bag file. The 'only' problem would indeed seem to be the sync between your video streams and other sensor data. If your capture system encodes a (wall) clock into your container then you should be able to sync everything using that. A small node that then decodes the video stream and uses the time code as the ROS timestamp could be an option. Or -- and probably more efficient -- use the embedded time codes to convert your video containers into bag files. That would all be off-line, so should reduce problems with resource usage that ...(more)
gvdhoorn: Yes, that is the most likely path we are going to take. For our later application we need random access to frames anyway so it would then make sense to extract images as separate files and skip the bags completely.