How can I train multiple reinforcement learning agents using a GPU in a Simulink environment in Matlab R2022a?

Good day,

I have implemented 3 PPO agents in a Simulink environment and successfully configured the training using the rlMultiAgentTrainingOptions. However, I am having problems implementing the training process using a GPU. There are Simulink examples available where RL agents can be trained on a GPU using the rlRepresentationOptions object and the UseDevice method , however this has since been deprecated and replaced by the rlOptimizerOptions object. I have also tried setting the UseDevice method of the rlRepresentationOptions to 'gpu' and using this instead of the newer rlOptimizerOptions object but this returned an error when used as input to the rlMultiAgentTrainingOptions method.

Therefore, my question is how can I implement GPU training on the rlMultiAgentTrainingOptions method when attempting to train multiple RL agents?

Best regards,

Brad

[Giancarlo Storti Gajani]

I have the same exact question !

Answer 1

[aditi bagora]

Hello Bradley,

I understand you want to train multiple reinforcement learning agents using a GPU in a Simulink environment. You have implemented agents and configured the training options using “rlMultiAgentTrainingOptions()”, which is now deprecated, and it is recommended to switch to “rlOptimizerOptions()” . However, you mentioned that setting the UseDevice option to "gpu" is not yielding the desired results.

To address this issue, I suggest the following approaches :

1. When creating the actor, make sure to set the UseDevice option to "gpu":

actor = rlDiscreteCategoricalActor( ___, UseDevice="gpu"); % Seting UseDevice parameter

2. Similarly, when creating the critic, ensure that the UseDevice option is set to "gpu":

critic = rlValueFunction( ___, UseDevice="gpu"); % Setting UseDEvice parameter

Please refer to the following documentation on using GPUs to train agent:

• Train agents using parallel computing and gpu.
• How to set UseDevice when actor is created.
• How to set UseDevice when critic is created.

Hope this information helps!

Regards,

Aditi