h5read being slow for files written by matlab

I have some raw data in hdf5 file format (lz4 compressed) this is really quick to read, whcih is nice ;-).

I then do some stuff to the data, and save it in new hdf5 files (noncompressed) which is unreasonable slow to read in comparison:

>> tic;rawpix = h5read('raw_data.h5','/h5/data/path',[1 1 1], [1 1 1]);toc      
Elapsed time is 0.154878 seconds.
>> tic;manpix = h5read('treated_data.h5','/h5/data/path', [1 1 1], [1 1 1]);toc    
Elapsed time is 0.400602 seconds.

The raw data file is 3.8GiB (including 8 billion uint32's)

the treated one is 5GiB (including 0.7 billion doubles)

Is there a way to get theese speeds the same?

I think lz4 compressing as I'm writing the files would solve the problem, But I don't know how to make matlab use lz4 commpresion while writing, as it's not documented on The h5write documentation site, any ideas?

Answer 1

[Preethi Ayyamperumal]

Currently, MATLAB supports only gzip compression for creating HDF5 dataset. Th compression levels are between 0-9.

Refer to the following link for applying gzip compression:https://www.mathworks.com/help/matlab/ref/h5create.html#buvazss-2

Please try this compression method to see if it resolves the performance issue.