Clear Filters
Clear Filters

Problem with RL agent block

19 views (last 30 days)
I have this simulink model which mainly utilizes functions blocks to process signal. The observation input for the RL agent is a scalar. obsinfo uses rlNumericSpec and ainfo uses rlFiniteSetSpec given its discrete property. I intend to validate the environment generated via the function env = rlSimulinkenv(mdl,agentblk,obsinfo,ainfo). To validate, I uses the function validateEnvironment(env). This generates an error.
The error states:
Error using rl.env.SimulinkEnvWithAgent/validateEnvironment. An error occurred while running the simulation for the model 'simulink2' with the following RL agent blocks: simulink2/RL agent
Model 'simulink2' has underspecified signal dimensions and Simulink has assigned these dimensions by default. To prevent ambiguities, consider speficying these dimensions in the block dialog box. Simulink has assigned these signal dimensions:
  • Block 'simulink2/RL Agent/last_observation' 'Output Port 1' has been set to [1]`
Before, I encounter 3 bullets with the same error. The two were pertaining to the same error regarding the specification of signal dimensions for last_action and external_action. This error were solved by enabling the RL agent parameter for this and assigning the appropriate signal values.
Now for the last_observation, I do not see how to access this parameter to assign the appropriate dimension in the RL agent block dialog box. What would be the way to let the model know that the dimension for the last_observation is indeed [1]? Is there a way to access this parameter in the block dialog box and where?

Accepted Answer

Emmanouil Tzorakoleftherakis
Edited: Emmanouil Tzorakoleftherakis on 24 Jan 2024
You can use a delay block for the last observation and set the initial value of the delay in the block dialog. That should resolve the dimension issue for this signal.
  1 Comment
Ethan Serapio
Ethan Serapio on 27 Jan 2024
I have solved the problem by adding a delay block in the model. Also, I have found in several forums that this is not an error but rather a warning although it appears as an error in the matlab window. The prompt may appear as an error given that it has a red font but in the default settings of the configuration in simulink, it is set as a warning. By this, the prompt can be disregarded by setting it as 'none' instead of 'warning'. The train function stops even for warnings thus the solution.
This option may not be practical for other models if the assumption of matlab with the dimension is incorrect, otherwise it is valid. This is also the solution for removing the matlab error regarding last_action and for the other one which I forgot which goes along with the error I've posted.
I have my RL model up and running. Thank you for your response.

Sign in to comment.

More Answers (0)

Products


Release

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!