Ask Your Question

Wait for images before prediction -- time steps LSTM -- callback issue

asked 2020-06-26 13:42:34 -0500

dr563105 gravatar image

updated 2020-06-26 13:43:50 -0500

Hey everybody!

I have implemented a LSTM deep learning model. I would like to use the trained model to predict autonomous driving commands. I use the LGSVL simulator to interact. During the autonomous driving part, I subscribe to 3 camera sensors and publish in a control command topic. Since they arrive at different rates, I use the ros2 message filters module and its ApproximateTimeSynchronizer function to make a callback.

In the callback, I predict the commands by using the images. Since I use LSTM, I have made my input images as time series sequences according to the timesteps. For example, if the timesteps = 2, the model expects 2 sets of 3 camera images to predict. So that I use both t and t-1 frame images to predict t+1. However, in callback only one set is received.

So my question is how can I wait till I get 2(changes with timesteps defined) sets of images to make a callback? Can I store the images in a list, call them when it reaches appropriate length for prediction?

Any ideas will be appreciated. Thank you.

edit retag flag offensive close merge delete


message_filters has a Cache. You can use that as you describe.

gvdhoorn gravatar image gvdhoorn  ( 2020-06-26 13:47:50 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted

answered 2020-06-30 14:07:39 -0500

dr563105 gravatar image

The timestamp I mention is the time at which that image is stored. When receiving from the node, I basically need to read it as a list and not through its timestamp.

So I implemented a FIFO queue to append at each callback and only invoke when the queue's length is equal to timesteps.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower


Asked: 2020-06-26 13:42:34 -0500

Seen: 17 times

Last updated: Jun 30