Ask Your Question

Writing messages to (individual) files

asked 2012-01-04 08:08:54 -0500

Mac gravatar image

I'm writing a small caching node, that takes in messages (which include an index) and provides a service that (given the index) returns the appropriate message.

I'm caching because storing all of these messages in memory is too expensive; my cache keeps a recent subset in memory, and writes the rest out to disk (and then does on-demand loading, should that be necessary).

I was doing this with a datatype that's easily serialized by hand, and uniquely-named files. I'm now adding more data to my cache type, and this will no longer work.

So: is there a straightforward way to say "Here's a message and a filename, dump the binary blob to that file"? Or is the correct way to generate a new bagfile (again, uniquely named) for each message I want to cache?

(Storing every message in one bagfile won't work, unless bags provide constant-time indexing, which I don't believe they do).

edit retag flag offensive close merge delete


What about just using the message serialization facilities for bagfiles instead of writing complete bagfiles?
dornhege gravatar image dornhege  ( 2012-01-04 08:12:31 -0500 )edit

2 Answers

Sort by » oldest newest most voted

answered 2012-01-06 09:50:39 -0500

bhaskara gravatar image

The warehousewg stack ( can be used for this — it uses a DB so you don't have direct control of the caching, but the DB will do it for you.

edit flag offensive delete link more


For posterity, writing one bag file per message (for my fairly large [~2MB]) messages worked fine, but this is the Right Way To Do It.
Mac gravatar image Mac  ( 2012-01-08 13:54:28 -0500 )edit

answered 2012-01-04 09:42:31 -0500

kwc gravatar image

You could attempt to invoke the serialization routines yourself and just write those blobs out to files of your choosing. This is how rosbag originally came about. In rospy this is trivial as its just a serialize() method on the message (though I am looking to move that). In roscpp, refer to:

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools


Asked: 2012-01-04 08:08:54 -0500

Seen: 884 times

Last updated: Jan 06 '12