MATLAB Answers

Saving Trained RL Agent after Training

84 views (last 30 days)
Hi All,
I trained a RL agent, the environment output was acceptable, my plan was to initially validate the agent in the simulation after training finished with the following code.
As i was concerned that I would restart training on the agent when I ran the script to run the 'sim' function, my IsDone flag in the simulation was manually set to 1 (previously 0 to permit training) and additionally commented out the 'training' function.
%trainingStats = train(agentSS,env,trainingOpts)
rng(0)
simOptions = rlSimulationOptions('MaxSteps',maxsteps);
experience = sim(env,agentSS,simOptions);
There was no ouput from the simulation, with no warnings, I then reset the IsDone flag back to 0, and reran the script, now the ouput was 0 on all scopes.
Did I lose the trained agent data when I set the IsDone flag to 1 after training?.
My next step was to try to save the trained agent with adding the following code found in the documentation, but still joy. My thoughts are I have overwritten and lost the trained data!
save("initialAgent.mat","agentSS")
load('initialAgent.mat')
rng(0)
simOptions = rlSimulationOptions('MaxSteps',maxsteps);
experience = sim(env,agentSS,simOptions);
How can I add code to ensure the trained agent data is saved automatically via 'RLTrainingOptions' after training has been completed, such as when maxepisodes are reached? Do not want to make the same mistake.
Is this correct?
trainingOpts = rlTrainingOptions(...
'MaxEpisodes',maxepisodes, ...
'MaxStepsPerEpisode',maxsteps, ...
'StopTrainingCriteria','AverageReward',...
'StopTrainingValue',-100,...
'ScoreAveragingWindowLength',100,...
'SaveAgentCriteria',"EpisodeCount",...
'SaveAgentValue',maxepisodes,...
'SaveAgentDirectory',"savedAgents")
Thanks
Patrick

Accepted Answer

Emmanouil Tzorakoleftherakis
Edited: Emmanouil Tzorakoleftherakis on 29 Apr 2021
Setting the IsDone flag to 1 does not erase the trained agent - it actually makes sense that the sim was not showing anything because it was immediately stopped by the IsDone flag.
To save the final agent, simply add the save command you have right after when you call 'train'.
My guess is that when you reran the whole script, you created a new agent from scratch and saved it again to a mat file, which replaced the already trained agent. This is why it's good practive to always have sections in your (live) script, so that you can pick exactly what lines you want to run.
  1 Comment
PB75
PB75 on 30 Apr 2021
Hi Emmanouil,
That worked, as you said I reran the whole script to run and validate the agent, hence, saved a fresh untrained agent. Running just the 'save', 'load' and 'sim' code section after training does the job.
Thanks

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!