HDF5 Library error

Hi,

I recently moved to matlab 2022a and without touching the code an error arrised. Indeed when trying to create a paraview file (h5), I get the following error. Is it a known issue of the new version ?

Error using hdf5lib2
Unable to open the file because of HDF5 Library error. Reason:Unknown

Error in H5F.open (line 130)
file_id = H5ML.hdf5lib2('H5Fopen', filename, flags, fapl, is_remote);

Error in h5write (line 108)
file_id    = H5F.open(Filename,flags,fapl);

Error in xdmf3writer>h5creation (line 85)
h5write(sprintf('%s.h5',filename),'/Geometry',Grid.geometry');

Error in xdmf3writer (line 13)
h5creation(filepath,Grid);

Error in save_xdmf (line 33)
xdmf3writer(sprintf('%s-%d',filename,id_model),Grids);

Error in paraview_save (line 49)
save_xdmf(model,result,file_name);

[Tyler Gross]

Any solution to this?

Experiencing this on R2021b as well using the generic Matlab provided examples on several systems with default installations.

[Stephen Schmugge]

Did you ever figure it out?

I'm experiencing the same problem. It seems to happen when I'm writing out tens of thousands of files (~43,000 files) in one run with multiple runs running at the same time on a HPC cluster. When I do not run the runs simultaneously on the cluster, it seems to not crash. Maybe matlab is not closing the file after I'm done writing? I see a close function ('https://www.mathworks.com/help/matlab/ref/fileh5f.html#mw_a1aaf05c-fd51-48a0-8671-7e5ac49d435d'), but I don't see where the fileID is returned so I can call it.

Here is the stack trace

Error using hdf5lib2
Unable to open the file because of HDF5 Library error. Reason:Unknown

Error in H5F.open (line 130)
file_id = H5ML.hdf5lib2('H5Fopen', filename, flags, fapl, is_remote);

Error in h5write (line 108)
    file_id    = H5F.open(Filename,flags,fapl);

Error in writeExampleCardiacEDF_D (line 13)
            h5write(exampleSavePath, '/example', currExample);

Error in generate_examplesCardiacEDF_D (line 290)
                    writeExampleCardiacEDF_D(outputExample, patientNum, currGTVector, timesVec, 0, exampleSaveDir, dayNum,partNum, iExample, dataSetType, currentDate, gtSeqVector);

Caught "std::exception" Exception message is:
OS_Rng: open /dev/urandom operation failed with error 24
Could not create on-disk crash report: failed opening file: Too many open files: iostream error

--------------------------------------------------------------------------------
             std::terminate() detected at 2023-03-31 00:18:25 -0400
--------------------------------------------------------------------------------

Configuration:
  Crash Decoding           : Disabled - No sandbox or build area path
  Crash Mode               : continue (default)
  Default Encoding         : UTF-8
  Deployed                 : false
  GNU C Library            : 2.28 stable
  Graphics Driver          : Uninitialized software
  Graphics card 1          : 0x102b ( 0x102b ) 0x536 Version 0.0.0.0 (0-0-0)
  Java Version             : Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
  MATLAB Architecture      : glnxa64
  MATLAB Entitlement ID    : 1302402
  MATLAB Root              : /apps/pkg/matlab/R2022b
  MATLAB Version           : 9.13.0.2049777 (R2022b)
  OpenGL                   : software
  Operating System         : "Red Hat Enterprise Linux release 8.3 (Ootpa)"
  Process ID               : 949606
  Processor ID             : x86 Family 6 Model 85 Stepping 7, GenuineIntel
  Session Key              : 64634000-362e-49b1-84f6-4326523d96d3
  Window System            : No active display

Fault Count: 1


Abnormal termination:
std::terminate()

Current Thread: 'MCR 0 interpret' id 22392415975168

Register State (captured):
  RAX = 0000000000000000  RBX = 0000145dee460328
  RCX = 0000000000000000  RDX = 0000145dee45a008
  RSP = 0000145da3ffa0c0  RBP = 0000145da3ffa4a0
  RSI = 0000145dee4402a1  RDI = 0000145da3ffa0d0

   R8 = 0000000000000000   R9 = 0000145da3ffa3b0
  R10 = 0000000000000000  R11 = 0000000000000000
  R12 = 0000145dee4703a8  R13 = 0000145da3ffa980
  R14 = 0000145da3ffa500  R15 = 0000000000000000

  RIP = 0000145dee3ee8cf  EFL = 0000000000000000

   CS = 0000   FS = 0000   GS = 0000

Stack Trace (captured):
[  0] 0x0000145dee3e6cb3     /apps/pkg/matlab/R2022b/bin/glnxa64/libmwfl.so+00322739




MATLAB is exiting because of fatal error
/var/spool/slurm/slurmd/job1137398/slurm_script: line 51: 949606 Killed                  /apps/pkg/matlab/R2022b/bin/matlab -nodisplay -singleCompThread $OPTS -r "inputIndex=${pNum}; ${sName};"

Answer 1

[Anurag Ojha]

Hi Alexandre,

There are three possible workarounds that can be followed to resolve this error.

• One way to solve the problem is by first collecting all the data blocks, keeping them in memory, and then combining them into a complete dataset. After that, save the full dataset to the HDF5 file using a single write operation. This approach helps avoid issues like truncation errors since the file size becomes maximum right away, and truncation is not involved.
• Another way to tackle the issue is by reading data blocks one by one. During the first iteration, create the file and also set up the last slab of the dataset with temporary values like zeros. This ensures the file reaches its maximum size in the initial iteration, preventing any additional truncation.
• The third workaround consists in using the low-level HDF5 interface along with the original sec2 low-level file driver instead of the VFS-based driver.

Hope this resolves your query!

[alexandre Daby-Seesaram]

Thanks !