how to obtain the data from the middle of the binary file using fread in MATLAB?
21 views (last 30 days)
Show older comments
I have a binary file with ~90 GB data with only single dimension. I would like to make a line spectra of the dataset in several chunks.
How to ask matlab to take the binary file ranging from (1:3e6) , ((3e6)+1:(3e6)*2) , and so on ... until the end of the file.
I've used:
fileID = fopen('Binarydata');
data1 = fread(fileID,[1:3e8],'uint16');
fclose(fileID);
plot(data1)
fileID = fopen('Binarydata');
data2 = fread(fileID,[(3e8)+1:(3e8)*2],'uint16');
fclose(fileID);
hold on; plot(data2)
But it throws error!
Error using fread
Invalid size.
Any help is appreciated
0 Comments
Answers (2)
Les Beckham
on 16 Nov 2023
Edited: Les Beckham
on 16 Nov 2023
The second argument to the fread function is the size of the data to read, not an index, as you appear to be trying to use it.
So, your code should change to something like the code below. Note the change to *uint16 in the precision argument of fread, so that Matlab won't convert the data to double, which is the default. This will use one fourth of the memory that would be used without the *.
Note also that I'm re-using the data array so you aren't keeping the old data in Matlab's memory when you read the second chunk of data.
Adapt as needed.
fileID = fopen('Binarydata');
data = fread(fileID, 3e8, '*uint16'); % read 300 million uint16 values as uint16 rather than doubles
% fclose(fileID); << you don't need to close the file if you are going to read from it again
plot(data)
% fileID = fopen('Binarydata'); << you don't need to re-open the file if you don't close it
data = fread(fileID, 3e8,'*uint16'); % read the next 300 million uint16 values
fclose(fileID);
hold on; plot(data)
Cris LaPierre
on 16 Nov 2023
The input to fread is the size of the output (i.e. the number of bytes to read), not a range of bytes. Try this.
fileID = fopen('Binarydata');
% read the first 3e8 bytes
data1 = fread(fileID,3e8,'uint16');
plot(data1)
% read the next 3e8 bytes
data2 = fread(fileID,3e8,'uint16');
hold on;
plot(data2)
hold off
fclose(fileID);
4 Comments
Walter Roberson
on 16 Nov 2023
The default when you specify a precision is that data is read in as that precision but converted to double automatically. But if you put * in front of the precision then data is read in as that precision and is kept in that precision instead of being converted to double.
So 'uint16' means to read unsigned 16 bit integers and convert them to double and return the doubles. But '*uint16' means to read unsigned 16 bit integers and leave them as unsigned 16 bit integers.
You can also use the '=>' specification to indicate what you want to be converted to. 'uint16' is equivalent to 'uint16=>double' and '*uint16' is equivalent to 'uint16=>uint16'.
See Also
Categories
Find more on Low-Level File I/O in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!