Revision history [back]

Hi,

It would help to see what command(s) you used to start the experiment.

My guess would be its because you need to provide a discretization for the value function in these continuous domains. You can do that with --nstates x option to discretize each dimension into x discrete states for the value function. On mountain car, I believe --nstates 10 or --nstates 20 work well. You can try this, or you can also try running an experiment on a discrete task such as taxi or tworooms and see how those go.

-Todd