Deep learning Toolbox - LSTM Training

40 views (last 30 days)
PB75
PB75 on 2 Sep 2022
Edited: PB75 on 7 Sep 2022
Hi All,
I am building an LSTM ROM to integrate into my Simscape model, which is using training and test data captured in ANSYS Chemkin software.
I have had multiple issues with pre-processing the data, which I am working through, that main reason being is ANSYS only uses a variable step solver.
I am basing my code on the LSTM ROM example. The validation of the LSTM model (before we integrate into the full Simscape model) will have to be a simple model using the "Stateful Predict" block, and then using the test and train data captured in the workspace to comapre to the LSTM model, using "From Workspace" array block to get the signals into Simulink.
I have been able to run the code, and the training progress seems that the model has minimised the RMSE and loss, however, when I try to run my simple validaton model the results show something else!, see below.
As I am unsure as to how to proceed with debugging, I am unsure whether it is the pre-processed data (which may need resampling), the LSTM network and parameter selection or the validation model that is the main cause.
My question is, can I now use the Deep Network Designer app to help debugging. Now using the code generated the concatenated data is now in a 10x1 cell, can I now save this as a datastore and use the app to train the network?
I can show the training progress, data structure and validation output below.
Training Progress:
Validation Model:
Validation Results: Test data top row, LSTM data bottom row.
Any suggestions on how to proceed would be great, as stuck as to how to proceed.
Thanks in advance,
Patrick

Answers (1)

Arkadiy Turevskiy
Arkadiy Turevskiy on 2 Sep 2022
Hi Patrick,
Two suggestions:
  1. Try to test your LSTM network in MATLAB first. Does it match the validation data. If it does, then the issue is with a Simulink model.
  2. If your validation data in Simulink does not start at time 0, you need to reset the state of LSTM in State and Predict block by putting this block into a resettable subsystem and triggering it before your data starts. Ie. if your data starts at t-0.2, then do a step from 0 to 1 at 0.1 sec and feed that into reset port of resettable susbystem to reset the state.
Arkadiy
  3 Comments
Ben
Ben on 6 Sep 2022
I'll add a few additional notes to Arkadiy's answer here to help debug further. A variation on 1. would be to try run the Simulink model with your training data, since you have an expectation of how that should perform from the training plot.
It may be useful to add additional outputs to the workspace exactly before the input and after the output of the Stateful Predict block. This will help check exactly what data is passed to the LSTM and that the LSTM outputs, and you can compare this to how the network was trained in MATLAB.
A few possible issues come to mind:
  • Arkadiy's point 2. on the LSTM state - it will only be reset at time 0, or when the stateful predict block is inside a resettable subsystem and a reset signal is received.
  • Simulink is taking different time steps to your data. The Rate Transition block has a NoOp label, but if you are using input data with uniform timesteps, and a variable step solver in Simulink, then I would expect this to be something else such as ZOH. This should be possible by setting the output port sample time of the Rate Transition block to match the uniform timesteps of your input data. Essentially the issue is that Simulink's variable step ODE solvers may take different timesteps to your input data which will corrupt the LSTM state. A potential alternative would be to swap Simulink's ODE solver to a fixed step ODE solver with step size matching your input data, but this may not be realistic when the ROM part of a larger system.
  • The validation data is simply too different to the training data and the model can't perform well on it. This will depend on the strategy used to split validation off from the initial dataset. You could check this by including the validation data during training - the plot will then contain the current loss on the validation data, which should roughly follow the training loss if the model is generalising well to the validation data.
Hope that helps,
Ben
PB75
PB75 on 7 Sep 2022
Edited: PB75 on 7 Sep 2022
Hi Ben,
Thanks for the extra guidance, so I have made a little progress with testing the network in a Simulink model, I will show the state predict block in my main model below and the ouput. My error seems to be the order of the inputs into the mux and sequenceinput. I do have an output from the state predict block now, but the results are poorer than those of the predict function?
Additionally, I was able to use the interp1 command and was able to resize the raw ANSYS data and effectively resample it, Resampling. So now I have all the data at a fixed time step of 2.0E-05 sec prior to post-processing, and now with a downsample factor of 5, which gives a reasonable step size of 1.0E-04 sec for the training and test data I think?
All my sample times in the state predict subsystem triggered block mirrors the sample rate used in the training and test of 1.0E-04 sec. Also I am using a fixed step solver.
I can test the network with the predict function, as show below. The output for the first output state is good, but the output of the remaining 3 states gets poor, which may mean I need to norm all the data prior to preparing the data. I have done a post for this, hope someone can help with my code errors for finding the maximum value and normalising the cell containing the data Normalising a cell. The fidelity of the predict function and state predict outputs seem vastly different, even with the same step sizes of 1.0E-04 sec.
Predict Function Ouput:
Simulink Model Output:
The top signal is the trigger signal, signals 2 are the model total volume and the triggered LSTM volume, signals 3, 4, 5 and 6 are the network ouput.
Simulink Model - sample rates set to 1.0E-04 sec as per LSTM training,and a fixed rate solver (ODE4)
Any suggestions on how to match the predict function output and state predict output would be great, and how to improve the network results in general.
Thanks again,
Patrick

Sign in to comment.

Products


Release

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!