Answered

Matlab reinforcement learning stop working after a while

125 episodes are not that many. Is it always freezing on that episode? Given that you are not getting any errors/crashes, my hun...

Matlab reinforcement learning stop working after a while

125 episodes are not that many. Is it always freezing on that episode? Given that you are not getting any errors/crashes, my hun...

6 månader ago | 0

Answered

DDPG with LSTM layer fails?

Hello, I see a couple of things wrong with the current architecture (could be more): 1) When you use the lstm layer, the input...

DDPG with LSTM layer fails?

Hello, I see a couple of things wrong with the current architecture (could be more): 1) When you use the lstm layer, the input...

6 månader ago | 0

| accepted

Answered

Could you help clarify the terminology and usage of Exploratory Policy and Exploratory Model in TD3 Reinforcement Learning

The answer above is correct with a small caveat: Even if UseExploratorypolicy is set = 0, the agent will still explore during t...

Could you help clarify the terminology and usage of Exploratory Policy and Exploratory Model in TD3 Reinforcement Learning

The answer above is correct with a small caveat: Even if UseExploratorypolicy is set = 0, the agent will still explore during t...

6 månader ago | 0

| accepted

Answered

Load a pretrained neural network object in rlNeuralNetworkEnvironment

Hi Vasu, You can use a pretrained environment model with MBPO agent as follows: 1) Create a rlContinuousDeterministicTransitio...

Load a pretrained neural network object in rlNeuralNetworkEnvironment

Hi Vasu, You can use a pretrained environment model with MBPO agent as follows: 1) Create a rlContinuousDeterministicTransitio...

6 månader ago | 0

Answered

Initial condition setting mPC

This should be helpful: https://www.mathworks.com/help/mpc/ref/mpcstate.html

Initial condition setting mPC

This should be helpful: https://www.mathworks.com/help/mpc/ref/mpcstate.html

7 månader ago | 0

Answered

is it possible to use neural network prediction model in Adaptive MPC ?

You should be able to do that, but we don't have an example to show the workflow unfortunately. You would first need to use aut...

is it possible to use neural network prediction model in Adaptive MPC ?

You should be able to do that, but we don't have an example to show the workflow unfortunately. You would first need to use aut...

7 månader ago | 0

| accepted

Answered

Make Jacobian as a function in Simulink

I would first use automatic differentiation to get the A,B,C,D matrices using this command. Then you would need to take the gen...

Make Jacobian as a function in Simulink

I would first use automatic differentiation to get the A,B,C,D matrices using this command. Then you would need to take the gen...

7 månader ago | 0

Answered

RL DDPG Actions have high oscillation

Hi, The noise options you are mentioning are only used during training and are essential for exploration. If the plots you are ...

RL DDPG Actions have high oscillation

Hi, The noise options you are mentioning are only used during training and are essential for exploration. If the plots you are ...

7 månader ago | 1

Answered

I want to know how to understand the sampling time ts of the agent in reinforcement learning, and how its value will affect the training and results?

Think of it as how often you need the agent to make decisions. Sometimes you need to make decisions very frequently (e.g. if you...

I want to know how to understand the sampling time ts of the agent in reinforcement learning, and how its value will affect the training and results?

Think of it as how often you need the agent to make decisions. Sometimes you need to make decisions very frequently (e.g. if you...

7 månader ago | 0

| accepted

Answered

An Error in LSTM Network Training Using Reinforcement Learning Toolbox

I believe that both the actor and the critic need to be LSTM networks. In your case only the critic is

An Error in LSTM Network Training Using Reinforcement Learning Toolbox

I believe that both the actor and the critic need to be LSTM networks. In your case only the critic is

7 månader ago | 0

| accepted

Answered

Linear Model Predictive Control with state feedback

The most straightforward way would be to assume that all your states are outputs, so set the C to be the identity matrix.

Linear Model Predictive Control with state feedback

The most straightforward way would be to assume that all your states are outputs, so set the C to be the identity matrix.

7 månader ago | 0

Answered

How to soften hard constraints on states in non-linear MPC?

Here is an example that shows how to do that with multi-stage MPC: https://www.mathworks.com/help/mpc/ug/use-multistage-mpc-wit...

How to soften hard constraints on states in non-linear MPC?

Here is an example that shows how to do that with multi-stage MPC: https://www.mathworks.com/help/mpc/ug/use-multistage-mpc-wit...

7 månader ago | 0

Answered

RL: Continuous action space, but within a desired range（use PPO)

You can always clip the agent output on the environment side. PPO is stochastic so the upper and lower limits are not guaranteed...

RL: Continuous action space, but within a desired range（use PPO)

You can always clip the agent output on the environment side. PPO is stochastic so the upper and lower limits are not guaranteed...

8 månader ago | 0

Answered

How to add custom environment for Reinforcement learning toolbox?

Why do you need a 3d world for this problem? Unless you consider the z dimension (e.g. if you do planning for UAVs), you only ne...

How to add custom environment for Reinforcement learning toolbox?

Why do you need a 3d world for this problem? Unless you consider the z dimension (e.g. if you do planning for UAVs), you only ne...

8 månader ago | 0

Answered

Reinforcement Learning - PPO agent with hybrid action space

Hello, The workaround you suggested makes sense to me. Unfortunately though, bullet #3 is not currently supported. You cannot d...

Reinforcement Learning - PPO agent with hybrid action space

Hello, The workaround you suggested makes sense to me. Unfortunately though, bullet #3 is not currently supported. You cannot d...

8 månader ago | 0

Answered

How to make the agent automatically pause after training to a certain number of turns

You can actually automate this process without having to stop training if you are using Simulink. There is an port 'Use external...

How to make the agent automatically pause after training to a certain number of turns

You can actually automate this process without having to stop training if you are using Simulink. There is an port 'Use external...

8 månader ago | 0

Answered

Scaling in the QP formulation

I think it's likely a typo. Scaling factors should be included in Wu, Wdu and Wy along with the individual weights.

Scaling in the QP formulation

I think it's likely a typo. Scaling factors should be included in Wu, Wdu and Wy along with the individual weights.

8 månader ago | 0

| accepted

Answered

For a Non linear Model predictive control toolbox, how can I give it a mathematical model, which is a subsystem defined in Simulink?

Unfortunately, this is currently not supported but our development team is aware of this request. If you don't have an analytica...

For a Non linear Model predictive control toolbox, how can I give it a mathematical model, which is a subsystem defined in Simulink?

Unfortunately, this is currently not supported but our development team is aware of this request. If you don't have an analytica...

8 månader ago | 0

| accepted

Answered

Constraints violation in MPC design

The more constraints you are adding, the smaller the solutions space, the harder it gets for the solver to find a solution. If y...

Constraints violation in MPC design

The more constraints you are adding, the smaller the solutions space, the harder it gets for the solver to find a solution. If y...

8 månader ago | 0

| accepted

Answered

Is there any way to graph the MPC cost function?

If you are designing the controller in MATLAB, all 'move' functions have an info argument which containts the cost value (see he...

Is there any way to graph the MPC cost function?

If you are designing the controller in MATLAB, all 'move' functions have an info argument which containts the cost value (see he...

8 månader ago | 0

Answered

The code for nmpc is not working even if i put it in same folder ?

I wasn't able to reproduce the error. Both state and output functions must be in different .m files. I did that and put everythi...

The code for nmpc is not working even if i put it in same folder ?

I wasn't able to reproduce the error. Both state and output functions must be in different .m files. I did that and put everythi...

8 månader ago | 1

| accepted

Answered

Problem with LSTM and PPO reinforcement learning

Hi, When you set up LSTM networks, the input layer needs to be the sequenceInput layer. See for example here: https://www.math...

Problem with LSTM and PPO reinforcement learning

Hi, When you set up LSTM networks, the input layer needs to be the sequenceInput layer. See for example here: https://www.math...

8 månader ago | 0

Answered

QP formulation from the MPC toolbox

We are currently using the dense formula as you mentioned, but also working on adding support for sparse problems. The following...

QP formulation from the MPC toolbox

We are currently using the dense formula as you mentioned, but also working on adding support for sparse problems. The following...

8 månader ago | 0

| accepted

Answered

Hierarchical Reinforcement Learning in Matlab

Hierarchical RL basically comes down to breaking a compelx problem down into smaller 'easier to train' problems, and potentially...

Hierarchical Reinforcement Learning in Matlab

Hierarchical RL basically comes down to breaking a compelx problem down into smaller 'easier to train' problems, and potentially...

8 månader ago | 0

Answered

Tune PI Controller Using Reinforcement Learning

I also replied to the other thread. The fullyConnectedPILayer is a custom layer provided in the example - you can open it and se...

Tune PI Controller Using Reinforcement Learning

I also replied to the other thread. The fullyConnectedPILayer is a custom layer provided in the example - you can open it and se...

8 månader ago | 0

| accepted

Answered

I want to convert the PI controller in this instance to a PID controller, so can my network be changed to something like this?

That should work - you are essentially adding one more weight to the PI layer for the D term. Potential issues may arise when yo...

I want to convert the PI controller in this instance to a PID controller, so can my network be changed to something like this?

That should work - you are essentially adding one more weight to the PI layer for the D term. Potential issues may arise when yo...

8 månader ago | 1

| accepted

Answered

Deep reinforcement learning and TD3 algorithm in the PID control

Have you seen this example?

Deep reinforcement learning and TD3 algorithm in the PID control

Have you seen this example?

8 månader ago | 0

| accepted

Answered

How do I stop the reinforcement learning training and ensure agent is saved when the Episode Manager Crashes. (Stop button on episode manager not accessible)

If you use the stop button, the agent you will get will have the parameters learned to that point. You can then continue trainin...

How do I stop the reinforcement learning training and ensure agent is saved when the Episode Manager Crashes. (Stop button on episode manager not accessible)

If you use the stop button, the agent you will get will have the parameters learned to that point. You can then continue trainin...

8 månader ago | 0

Answered

How could I realize from the graph of "reinforcement learning episode manager" in DDPG or rlTD3Agent that the network is learning well?

You should be seeing the average reward curve going upwards (not monotonically) which is not the case here.

How could I realize from the graph of "reinforcement learning episode manager" in DDPG or rlTD3Agent that the network is learning well?

You should be seeing the average reward curve going upwards (not monotonically) which is not the case here.

8 månader ago | 0

| accepted

Answered

How do i define and extract the MPC constraints?

The getconstraint function is supposed to return mixed input/output constraints. One use case would be that someone creates the ...

How do i define and extract the MPC constraints?

The getconstraint function is supposed to return mixed input/output constraints. One use case would be that someone creates the ...

8 månader ago | 0