appending variables in save causing file corruption

12 views (last 30 days)
Hi,
I have some .mat files containing several variables of type 'Table'. I perform some analysis and replace some of these variables using the command
save('FileName', {'VarX', 'VarY', etc}, 'mat', '-append')
In this case there are other variables in the file appart from VarX and VarY which I am not overwriting. this seems to work most of the time, but every now and again i get an error:
Error using save
Error closing file 'FileName'
The file may be corrupt.
The files I am dealing with are around 250-500MB in size.
Any ideas why this might happen? It means I then corrupt my original data which is annoying. Perhaps I am best to create a totally new file with the new data and then replace the old file, therefore if I do get a corrupt file when writing I won't have destroyed my original data.
Thanks
Oli.
  2 Comments
Robert Reid
Robert Reid on 17 Apr 2021
Edited: Robert Reid on 17 Apr 2021
Hi Oli,
I think that I am running into the exact same problem - your description at least matches my issue exactly.
Did you ever find a good solution to this?
Just saw Sai's answer below - did matfile help solve this issue?
Thanks,
Rob
Oliver Warlow
Oliver Warlow on 18 Apr 2021
Hi Rob,
I have to say I think i just implemented a rather memory wastful method of loading the complete file and resaving rather than appending the new data. I think that the problem was actually related to the files being located on a local network drive rather than local on my machine, I found this was causing corruption issues, to get round this agains I implemented not a very efficienct work around which was to save the file locally and then copy it to its required location. I never actually tried using the matfile method as I think I had already implemented my work around.
sorry for not being much more help. I have now changed the way I manage this data storing it as tales on an SQL server instead of as .mat files
O.

Sign in to comment.

Answers (1)

Sai Bhargav Avula
Sai Bhargav Avula on 6 Nov 2019
Edited: Sai Bhargav Avula on 6 Nov 2019
Hi,
One reason that the files get corrupted is because you might be trying to overwrite it before it could save the old process.
Also I would recommend you to use the matfile, as it can access and change the variables without loading it into the memory. Because of this the MAT file object requires less memory than the load and save commands.
Hope this helps!

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!