Using matfile to partially extra data still loads entire file into memory
10 views (last 30 days)
Show older comments
Hello,
I have a .mat file saved in the -7.3 format. The content of the file is a large cell array. I am using (,) indexing to retrieve a single row:
obj = matfile('File.mat');
Data = obj.CellArray(RowNum,:);
I've done some investigating on the memory usage and when I run that command, it works, I get the line out of the cell array.
However, it takes the same amount of time as loading the .mat file into the workspace and it uses the same amount of memory. From the MATLAB help files I thought this syntax was designed to only partially load files into memory. Am I doing something incorrect or does the feature not work the way I hoped it would?
Thank you for your help.
3 Comments
dpb
on 8 Apr 2021
I guess with cell arrays I'm not terribly surprised that it might have to dereference them to get stuff out -- and so, while what it returns is only what is asked for, it took the same or more effort to produce than just the straight load and then clearing what don't want.
In straight arrays, the direct location can be computed and memcopy() invoked on a buffer and stuff can be streamlined; I've no idea what the actual memory structure of cell arrays is having never poked around in the innards, but there's a whole lot of overhead associated with them and tables add yet another layer on top.
Answers (2)
Matt J
on 9 Apr 2021
Edited: Matt J
on 9 Apr 2021
We can run a test right here. The one below suggests there is some benefit, though perhaps not as much benefit as I would have expected given the size of the data being loaded. You're sure the format of your File.mat is v7.3?
CellArray(1:100,1:100)={rand(50)};
save -v7.3 File CellArray
tic;
L=load('File');
toc
obj=matfile('File.mat','Writable',false);
tic
obj.CellArray(20,:);
toc
See Also
Categories
Find more on Workspace Variables and MAT-Files in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!