Using environment reset function: Access Simulink workspace for if then tests
Show older comments
I'm attempting to code a PPO RL agent in which a glider is released and attempts to navigate to a target within "R997" distance. I've generated the model within Simulink including outputing the last distance value to the target ("D2Tgt") with a workspace block. For navigating to the target within a R997 value, it gets a reward. It gets other reward and penalties along the way as well. Some publications have mentioned such training cases can be speed up by holding the target location constant instead of randomizing it each training episode until the it starts getting close where you start randomizing it. This is done by starting with a large acceptable distance and decreasing it over the training time. This is what I've attempted to code below. The issue I'm having is pulling the "ans.D2Tgt" from the workspace during RL training even though I have a simulink workspace out block for this value. Any advice???
ans.D2Tgt=random initalization for now
env = rlSimulinkEnv('SimulinkModel','SimulinkModel/RL Agent',obsInfo,actInfo);
env.ResetFcn = @(in) localResetFcn(in, R997, ans);
function in = localResetFcn(in, R997, ans)
persistent n
if isempty(n)
n=0
end
ans.D2Tgt=ans.D2Tgt
DropHeight = randi([deleted values were here],1,1);
blk = 'SimulinkModel/6DOF Wind (Wind Angles)';
in = setBlockParameter(in,blk,'xme_0',mat2str([0 0 -DropHeight]));
ReleaseVelocity=randi([deleted values were here],1,1);
blk = 'SimulinkModel/6DOF Wind (Wind Angles)';
in = setBlockParameter(in,blk,'Vm_0',mat2str([ReleaseVelocity 0 0]));
if (ans.D2Tgt <= R997 * 5) && (n==0)
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
in = setBlockParameter(in,blk,'const',mat2str(R997 * 4));
elseif (ans.D2Tgt <= R997 * 4) && (n==1)
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
in = setBlockParameter(in,blk,'const',mat2str(R997 * 3));
elseif (ans.D2Tgt <= R997 * 3) && n==2
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
in = setBlockParameter(in,blk,'const',mat2str(R997 * 2));
elseif (ans.D2Tgt <= R997 * 2) && n==3
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
in = setBlockParameter(in,blk,'const',mat2str(R997 * 1));
elseif (ans.D2Tgt <= R997 * 1) && (n>=4)
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
else
end
end
1 Comment
Emmanouil Tzorakoleftherakis
on 21 Dec 2023
Apologies, I am not following the question. Are you able to use D2Tgt inside the reset function? If yes, where is the problem exactly?
Answers (0)
Categories
Find more on Reinforcement Learning Toolbox in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!