Training CNNs on video data

I have some video files (in .mat format). these files are time series 3D reconstructions (from digital holograms) of blood cells (two classes: healthy and diseased cells). each frame contains spatiotemporal membrane fluctuations (of the order of tens of nanometers). Is it possible to train the CNNs on these .mat files? I know that training CNNs requires a ton of data (which I don't have), but I can generate some synthetic data, is this the norm? I am thinking of using the same CNN model as used in digit classification: https://www.mathworks.com/help/nnet/ref/trainnetwork.html

Answer 1

[Kenta]

One option is to use CNN and LSTM. As long as the data is correctly saved in the mat file, you do not have to care about the file format. The below is a demo of video classification which may relate to your study.

https://jp.mathworks.com/matlabcentral/fileexchange/74402-video-classification-using-lstm-lstm

The image features were extracted via a pre-trained network and the time-series features were classified using LSTM (Long Short Term Memory). A netowrk trained with ImageNet was used for a future extractor, but the domain you focus is different, then, you might update the parameters just like fine-tuning.