- Your arrays should be 2D with a size of [8, 24]. Use MATLAB's "size" function to verify the correct array structure.
- The step function should return observations that align with your updated specification, i.e., 2D arrays of size [8, 24].
- If you're using a neural network or similar for your Q-function, it may need to be updated to handle the increased dimensionality of the observation space.
- Use "validateEnvironment" function in MATLAB to check if your environment meets the requirements for reinforcement learning.
Observation for Reinforcement Learning as Array
4 views (last 30 days)
Show older comments
Hello,
I have created a reinforcement learning environment. It is a system in which the agent always has to make decisions (actions) for the following hour. The basis (observations) for the "decisions" (actions) are forecast values that are taken from time series. In the current version, only the forecasts for the following hour are transferred.
ObservationInfo = rlNumericSpec([8 1]);
ObservationInfo.Name = 'Observation';
ObservationInfo.Description = 'Load_obs, PV_gen_obs, batt_obs, Price_obs, Heat_demand_obs, Heat_stor_obs, hour_of_day_obs,Compense_obs';
For some of the values, however, not only the forecast value for the coming hour, but also for the following hours should be transferred (e.g. 24h).
My approach was as follows:
ObservationInfo = rlNumericSpec([8 24]); % Not 8 individual values but 8 arrays with 24 values each are transferred.
ObservationInfo.Name = 'Observation';
ObservationInfo.Description = 'Load_obs, PV_gen_obs, batt_obs, Price_obs, Heat_demand_obs, Heat_stor_obs, hour_of_day_obs,Compense_obs';
I have checked the step function and the arrays are created as required. However, I cannot create an executable environment. Individual values are still passed as individual values. I have passed these into an array with the value to be passed at position 1 and 23 zeros.
Does anyone have any suggestions?
BR
Felix
0 Comments
Answers (1)
Himanshu
on 28 Sep 2023
Hello Felix,
I understand that you are trying to modify the observation space of your reinforcement learning environment to include 24-hour forecasts for certain variables. The issue seems to be that the environment is not executing as expected, possibly due to how the observations are passed into the environment.
You can follow the below steps to fix the issue:
You can refer to the below documentation to learn more about Creating Custom Reinforcement Learning Environments in MATLAB. Also, please refer to the documentations to know more about the "rlNumericSpec" and the "validateEnvironment" functions in MATLAB.
0 Comments
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!