Can I use rlNumericSpec for GridWorld Env for Action Space

In my previous question (https://www.mathworks.com/matlabcentral/answers/550113-specifying-actions-using-rlfinitesetspec-or-rlnumericspec-reinforcement-learning#answer_453069?s_tid=prof_contriblnk) I was told that I should use rlFiniteSetSpec for the U,D,L,R actions of an agent in a GridWorld Environment.

My follow up is can I use rlNumericSpec where I define two outputs: one for xmovement, one for ymovement. Then I would define my actInfo as the following:

actInfo = rlNumericSpec([-5 5; -5 5])

where -5 5; in both rows represent the fact that I can move in both xy-direction any step of units from -5 to 5. Is that allowed?

Answer 1

[Madhav Thakker]

Hi Huzaifah,

I understand that you are able to run grid world for discrete U, L, D, R actions and are trying it with continuous action space.

The correct way of creating a continuous action space with your requirements is -

actInfo = rlNumericSpec([2 1], 'LowerLimit', -5, 'UpperLimit' ,5);

But, as per my understanding, it is not possible to use continuous action space in grid world with MATLAB. You can however create different discrete actions by creating custom grid world environments, particularly the Actions property of the grid world model.

For example,

createGridWorld(5,5, 'Kings') % creates a grid world with 8 different actions. 

Hope this helps.