large csv import and split

7 views (last 30 days)
Ancalagon8
Ancalagon8 on 10 Feb 2018
Answered: Jeremy Hughes on 12 Feb 2018
As a newbie i am working on a signal retrieved from a geophone. What i am trying to do is to import my .csv file (weekly file size around 25GB) into matlab in order to process it. Microsoft excel doesn't open it of course. One solution is to split the original .csv into smaller, but i prefer to plot all the file at once (if it is possible).
Αs shown below, first column contains date&time, second column yes/no from a rain sensor and last column voltage.
"2018-01-25T13:17:46.203005" "NO" "0.05"
How can i import my initial large .csv file into 3 different columns (matrices, not cells)? Thanks in advance!

Answers (1)

Jeremy Hughes
Jeremy Hughes on 12 Feb 2018
With a file that's 25GB, you're going to need to break the data up. I recommend using
ds = tabularTextDatastore(filename)
while hasdata(ds)
T = read(ds)
end
You might have to tweak some settings, but this is the easiest way.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!