How to handle big csv files
3 views (last 30 days)
Show older comments
I have huge .csv files comming daily and i want to know how i can handle them. For the moment, only to have an idea, i run a python script first in order to decrease the size by averaging the file. Is splitting the big file the only option i have?
P.S. one line of my file looks like:
00:00:0;0.0;0.00540500736377
0 Comments
Answers (1)
Walter Roberson
on 1 Sep 2018
Another approach is to fopen() the file, and textscan() using a repeat count (a number as the parameter after the format string) to control how many lines are read at a time; you would read a chunk, process it, go back to read another chunk, and so on.
0 Comments
See Also
Categories
Find more on Standard File Formats in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!