Plotting a Simulink reinforcement learning environment

Hello there,

I have declared an environment starting from a simulink model:

env = rlSimulinkEnv("main","main/RL Agent",obsInfo,actInfo);

Then i have properly defined the training options and i have executed the training.

Now, I would like to plot the environment during the training process. In the help center i have found the possibility to use create a plot function exploiting a callback on environment update.

Are there any ways to do something similar also for the rlSimulinkEnv command without defining a custom environment with all the dynamic equations?

Thank you in advance.

Answer 1

[Shubham]

Hi Leonardo,

In MATLAB's Reinforcement Learning Toolbox, visualizing the environment during training can be quite useful for understanding the agent's behavior and progress. While the toolbox provides a straightforward way to set up environments using rlSimulinkEnv, it does not directly offer built-in callbacks for plotting during training. However, you can achieve this by defining a custom plot function and integrating it into the training process.

Here's a general approach to visualize the environment during training using rlSimulinkEnv:

1. Create a function that will handle the plotting of your environment. This function should be able to access the necessary state information to update the plot dynamically.

function myPlotFunction(agent, environment)
    % Example: Plot some aspect of the environment or agent
    % Retrieve state information from the environment
    state = environment.getObservation();
    % Update your plot here
    % e.g., plot(state(1), state(2), 'o');
    drawnow;
end

2. Instead of using the built-in training loop, you can create a custom training loop that calls your plot function at each step.

% Create the environment
env = rlSimulinkEnv("main","main/RL Agent",obsInfo,actInfo);

% Define the agent and training options
% (Assuming you have already done this part)

% Custom training loop
maxEpisodes = 1000; % Define the number of episodes
maxStepsPerEpisode = 500; % Define the max steps per episode

for episode = 1:maxEpisodes
    % Reset the environment at the start of each episode
    observation = reset(env);
    done = false;
    stepCount = 0;

    while ~done && stepCount < maxStepsPerEpisode
        % Get action from the agent
        action = getAction(agent, observation);

        % Step the environment
        [nextObservation, reward, done, ~] = step(env, action);

        % Update the agent
        agent = agent.learn(observation, action, reward, nextObservation, done);

        % Call the custom plot function
        myPlotFunction(agent, env);

        % Update observation
        observation = nextObservation;

        % Increment step count
        stepCount = stepCount + 1;
    end
end

3. Use Callbacks (Optional): If you prefer to use callbacks, you can define a callback function that updates the plot and pass it as part of the training options. However, this requires more advanced manipulation and is not directly supported for rlSimulinkEnv.

This method allows you to visualize the environment's state or any other relevant information during training without needing to define a custom environment with all dynamic equations. You can adjust the myPlotFunction to suit your specific visualization needs.