can i decide the RL agents actions
7 views (last 30 days)
Show older comments
I am training a PPO agent and issue is it keeps on searching for a better value even after reaching close to stable state.
what i mean is I want my agent to keep applying last action values as soon as the error values reaches <= 0.05 (to prevent oscillations and offset near the set point as shown in shared image.)
my question is can i do it in matlab because i know you can do it in python for sure. any help would be really really helpfull :)
3 Comments
Answers (2)
Sam Chak
on 4 Sep 2023
Hi @Sourabh
I believe that it has something to do with the StopTrainingCriteria and StopTrainingValue options of your rlTrainingOptions object. Is the condition "steady-state error ≤ 0.05" reflected in the training termination condition? Typically, the agent will continue to train until MaxEpisodes is reached when the stopping condition is not satisfied.
maxepisodes = 6000;
maxsteps = 150;
trainingOpts = rlTrainingOptions(...
'MaxEpisodes', maxepisodes,...
'MaxStepsPerEpisode', maxsteps,...
'ScoreAveragingWindowLength', 5, ...
'Verbose', false,...
'Plots', 'training-progress',...
'StopTrainingCriteria', 'AverageReward',...
'StopTrainingValue', 1500);
Also, please note that the rewards obtained by the final agents are not necessarily the greatest achieved during the training episodes. You need to save the agents that meet the "steady-state error ≤ 0.05" condition during training by specifying the SaveAgentCriteria and SaveAgentValue properties in the rlTrainingOptions object.
See also:
Emmanouil Tzorakoleftherakis
on 25 Sep 2023
Edited: Emmanouil Tzorakoleftherakis
on 25 Sep 2023
It seems like the paper you saw uses some logic to implement the behavior you mention. You could do the same with an if statement in MATLAB.
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!