How to read binary file from PDIR-NOW data?

I am having trouble opening a .bin file which the data provider highlight is in format "2-bytes" ("int16" in Matlab) format in little-endian byte-order. The data is a matrix with size of 3000x9000.

I just used the fopen() and fread() functions, but I got this kind of 'discontinuities' in the vertical axis.

gunzip('...\pdirnow1h20120922.bin.gz');

fid = fopen('...\pdirnow1h20120922.bin','r');

format = 'int16';

data = fread(fid, [3000 9000],format,'ieee-le');

imshow(data);

I dismiss the fact that the data is wrong because i already opened in python without problem, however, i need to process the data in matlab.

with open('/pdirnow1h20120922.bin', 'rb') as f:

data = np.fromfile(f, dtype='<i2')

data = data.reshape(3000,9000)/100

plt.imshow(data)

Any ideia of what i did wrong?

Pd. i am getting the data from this site: https://chrsdata.eng.uci.edu/ and users intrctions are in: https://persiann.eng.uci.edu/CHRSdata/PDIRNow/PDIRNow_readme.txt

Answer 1

[dpb]

Attach the file itself; don't expect folks go offsite somewhere to try to retrieve your data for you...

But, I did go look (not sure how many others may) and it looks to me like

fid = fopen('...\pdirnow1h20120922.bin','r');
data = fread(fid,[9000,3000],'int16').'; 
fid=fclose(fid); clear fid
imshow(data);

should be all need; MATLAB defaults to little-endian byte order.

NOTA BENE: the reference notes that data were written in row-major order; "Data are stored in the file in row-major orientation with the first 9000 values the N-most row...".

You read 3000x9000 instead; that introduces the discontinuity. MATLAB is column-major storage so need to transpose the input...and, while it's simply scaling, the above doesn't include the x100 scale factor you used in the other code.

Or, you can simply read the full file into the vector and reshape...

fid = fopen('...\pdirnow1h20120922.bin','r');
data = reshape(fread(fid,'int16'),9000,[]).'; 
fid=fclose(fid); clear fid
imshow(data);

[Luis castillo]

Thanks for your comment, another way to solve the problem:

According to the data provider, the issue is due to matlab is column centric, this means that It fills column by column rather than row by row. So the fix the issue is pretty simple such as:

data = fread(fid,[9000 3000],'int16');

data = data';

[dpb]

If you'll note, that's precisely the first solution I gave you in one line of code instead of two -- and using the "dot" transpose operator (.') for real data rather than the complex conjugate operator.

For pedagogical purposes I also showed an alternate syntax more akin to the other code you posted that scarfs up the file as a vector first.