How to properly set up my data to train an LSTM network

Hi there, I have time series data from some robot logs that I have run that have accelerometer data and motor speed data that I plan to use to train an LSTM to determine if the robot is stuck. I currently have the data broken up into text logs that store each data signal over a time range of the acceleration of the robot from stop to full speed. The text file is also tagged wether it is stuck or not so that it could check the prediction. Now, what is the best way to get this data in to use? It seems like it needs to be a data store object. Would it make sense to just read in each text file and save each as a separate cell array then in that cell array create a separate column in that cell array with the pass/fail status?

The Mathworks tutorials are useful but they seem to gloss over actually transitioning their models to data not prepared by them.

Thanks! I appreciate it.

Answer 1

[Sandeep]

Hi Wilson,

It is my understanding that you are expecting to find an approach to process the data collected from the robot legs and form appropriate dataset to train your LSTM model. Yes, you can read in each text file and save each as a separate cell array and then add a column to the cell array to specify the pass/fail status. To do this, the textscan function in MATLAB can be used to read data from a text file.

A Sample implementation is given below,

filename = 'robot_log.txt';  % replace with the name of your text file
fileID = fopen(filename,'r');
C = textscan(fileID,...
        '%f %f %f %f %f %f %f %s', ...
        'HeaderLines',3,'Delimiter','\t');
fclose(fileID);

data = [C{1} C{2} C{3} C{4} C{5} C{6} C{7}];

status = C{8};
status = cellfun(@(x) strcmp(x, 'pass'), status); % convert "pass" to logical true
status = [status ~status]; % add column for "fail"

The textscan function reads in the data from the text file and store it in a cell array "C". The columns containing the accelerometer and motor speed data are then concatenated into a single array data.

You can then repeat this process to read in the data from each text file and save it as a separate cell array with the pass/fail status included. Once you have all the data stored in cell arrays, you can concatenate them into a single array if that is more convenient for subsequent processing.