How to store large data sets with different types of information?

5 views (last 30 days)
I was wondering if it were possible for me to store data that I organize / analyze in MatLab. I'll describe the type of data, but there is a lot of it and I don't know how to store or manage it within MatLab.
I am doing data related to Environmental Engineering, essentially I have ~ 350 watersheds that I get data from. Within each watershed, there I have a daily precipitation / runoff information: a vector of the date that is a datetime and in corresponding rows I also have a vector of precipitation and a vector of runoff.
I hope to access the data within each watershed and having all of it in one place instead of making a variable file for each of the 350 watersheds, I have a vector with their site numbers. And within each watershed I would want a way to access the data of each day (the date, precipitation, and rainfall).
Please send any ideas or questions for more clarification.

Accepted Answer

Bjorn Gustavsson
Bjorn Gustavsson on 4 Sep 2019
If you're planning to run all analysis in matlab why not just store everything in a .mat-file. In this scenario just have a sit-down sketch out your data-types. Perhaps an array of structs, with each watershed in one element, with fields for each part. Something like this:
Watersheds(1).T_obs = time_obs;
Watersheds(1).precipitation = prec_daily;
Watersheds(1).runoff = runoff;
...etc...
Or you could save everything in cell-arrays. If every watershed has same number of datapoints for all elements it might be better to save everything into bigger matrices.
If you intend to share your data with people working with other programming environments you might have a look into hdf5-files or net-cdf files for general accessibility (my experience with data in such file-formats are "mixed at best", the documentation of the data-structure always seem to be missing or confusing).
My 5p's worth of thoughts.

More Answers (0)

Categories

Find more on Data Import and Export in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!