How do I properly substitute rlRepresentation with rlValueRepresentation, rlQValueRepresentation, rlDetermin​isticActor​Representa​tion, and rlStochast​icActorRep​resentatio​n?

I am using MATLAB r2020a where rlRepresentation is "not recommended." As a result, I am forced to substitute it with the criritics or actors in the following compatibility guide (https://www.mathworks.com/help/reinforcement-learning/ref/rlrepresentation.html#mw_a6277225-fecf-4d97-9549-1fc4799bf5b6). I tried replacing rlRepresentation with rlValueRepresentation, rlQValueRepresentation, rlDeterministicActorRepresentation, and rlStochasticActorRepresentation (though I left rlRepresentationOptions as is where it came up). They all resulted in errors, and rlValueRepresentation and rlStochasticActorRepresentation had the fewest (and the same) errors:

Error using rlStochasticActorRepresentation (line 93)

Too many input arguments.

Error in createDDPGNetworks (line 51)

critic = rlStochasticActorRepresentation (criticNetwork,criticOptions, ...

Since both this critic and actor have the same error, I think it might have something to do with rlRepresentationOptions since it gives properties to the actors or critics (as far as I understand).

For reference, I am trying to emulate this project (https://www.youtube.com/watch?v=6DL5M9b2j6I) in MATLAB r2020a.

Any help is appreciated.

[Giampiero Campa]

The table here might help too:

https://www.mathworks.com/help/reinforcement-learning/release-notes.html#mw_b16831a9-5f9d-47ab-a9c3-450cb6ceab15

[Salma Khaled]

Have you reached a solution?

Answer 1

[Emmanouil Tzorakoleftherakis]

It would be helpful if you pasted the exact MATLAB code you are typing to see what the problem is. I suspect you simply changed the method name, which is why you get the error you are seeing. Have a look at the documentation page for the respective method you want to use (rlValueRepresentation etc) and make sure the order and number of arguments matches the doc.

Answer 2

[Salma Khaled]

1. Create an actor using an rlDeterministicActorRepresentation object.
2. Create a critic using an rlQValueRepresentation object

https://www.mathworks.com/help/reinforcement-learning/ug/ddpg-agents.html

Answer 3

[Giampiero Campa]

The table in this page might help.