ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question
0

Node Agent texplore "killed", using reinforcement learning pkg

asked 2013-03-12 15:57:23 -0500

LeonardoLeottau gravatar image

Hi, I using ROS fuerte, ubuntu 11.10 and I am testing the RL pkg. I am runing texplore as option for the agent node and I have been tried to run different environmens but the node is killed before one or two episodes. Below I attach the terminal ouput of the agent node, this is run with mcar stocastic environment.

I have tried with car72, car27, carranddom, mcar, and several planners and models with the same problem.

Thanks.

agent: 

texplore
model: M5 Tree
planner: Parallel Real-Valued UCT
[ INFO] [1363138771.925508025]: RLAgent: starting main loop
Agent: Model Based
Agent: Model Based
feature 0 has range 1.8
feature 1 has range 0.14
reward range: 1
Parallel ETUCT tracking real state values
Planner using history size: 0
Start model learning thread
start parallel uct planning search thread
Created MultClass 0 with nModels: 5, addNoise: 0
Created m5 decision tree 0 multivariate regression (all feats) MIN_SDR: 0.00018
Created m5 decision tree 6 multivariate regression (all feats) MIN_SDR: 0.00018
Created m5 decision tree 12 multivariate regression (all feats) MIN_SDR: 0.00018
Created m5 decision tree 18 multivariate regression (all feats) MIN_SDR: 0.00018
Created m5 decision tree 24 multivariate regression (all feats) MIN_SDR: 0.00018
Created MultClass 2 with nModels: 5, addNoise: 0
Created m5 decision tree 2 multivariate regression (all feats) MIN_SDR: 0.0001
Created m5 decision tree 8 multivariate regression (all feats) MIN_SDR: 0.0001
Created m5 decision tree 14 multivariate regression (all feats) MIN_SDR: 0.0001
Created m5 decision tree 20 multivariate regression (all feats) MIN_SDR: 0.0001
Created m5 decision tree 26 multivariate regression (all feats) MIN_SDR: 0.0001
Created MultClass 3 with nModels: 5, addNoise: 0
Created m5 decision tree 3 multivariate regression (all feats) MIN_SDR: 0.0001
Created m5 decision tree 9 multivariate regression (all feats) MIN_SDR: 0.0001
Created m5 decision tree 15 multivariate regression (all feats) MIN_SDR: 0.0001
Created m5 decision tree 21 multivariate regression (all feats) MIN_SDR: 0.0001
Created m5 decision tree 27 multivariate regression (all feats) MIN_SDR: 0.0001
Created MultClass 1 with nModels: 5, addNoise: 0
Created m5 decision tree 1 multivariate regression (all feats) MIN_SDR: 1.4e-05
Created m5 decision tree 7 multivariate regression (all feats) MIN_SDR: 1.4e-05
Created m5 decision tree 13 multivariate regression (all feats) MIN_SDR: 1.4e-05
Created m5 decision tree 19 multivariate regression (all feats) MIN_SDR: 1.4e-05
Created m5 decision tree 25 multivariate regression (all feats) MIN_SDR: 1.4e-05
Episode 0 reward: -44
Killed
dlf@dlf-Satellite-L745:~/fuerte_workspace/sandbox/rl-texplore-ros-pkg/stacks/reinforcement_learning/rl_env$
edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
0

answered 2013-03-13 06:13:25 -0500

toddhester gravatar image

Hi,

It would help to see what command(s) you used to start the experiment.

My guess would be its because you need to provide a discretization for the value function in these continuous domains. You can do that with --nstates x option to discretize each dimension into x discrete states for the value function. On mountain car, I believe --nstates 10 or --nstates 20 work well. You can try this, or you can also try running an experiment on a discrete task such as taxi or tworooms and see how those go.

-Todd

edit flag offensive delete link more
0

answered 2013-03-13 09:09:32 -0500

LeonardoLeottau gravatar image

Great, with --nstates 10 it works fine. I am using these commands:

rosrun rl_env env --env mcar --stochastic --prints
rosrun rl_agent agent --agent texplore --model m5tree --planner parallel-UCT  --nstates 10 --prints

Another question: Algorithms as Filtered Rmax or Qlearning with tile coding could be run modifying the node options? Thanks again.

edit flag offensive delete link more

Question Tools

Stats

Asked: 2013-03-12 15:57:23 -0500

Seen: 220 times

Last updated: Mar 13 '13