MATLAB Answers

data loop for matlab

3 views (last 30 days)
zakary Surprenant
zakary Surprenant on 17 Nov 2020
Edited: Adam Danz on 8 Dec 2020
for i= 1977:2012
for j=1:12
for k=1:31
mx=min(q1,[],'omitnan');
end
end
end

  4 Comments

Show 1 older comment
zakary Surprenant
zakary Surprenant on 18 Nov 2020
I forgot to mention that I want to find the max values for each month!
Adam Danz
Adam Danz on 18 Nov 2020
How are your data arranged? Could you show us a sample?

Sign in to comment.

Accepted Answer

Adam Danz
Adam Danz on 18 Nov 2020
  1. Read data in as a table (see readtable)
  2. Convert the year-month-day columns to a single datetime vector and convert the table to a timetable
  3. Use retime to get the monthly max
Demo:
% Create demo table
T = table([1977;1977;1977;1977;1978;1978],[11;11;12;12;1;1],[1;2;1;2;1;2],(2:7)','VariableNames',{'Year','Month','Day','MaxTemp'})
T = 6x4 table
Year Month Day MaxTemp ____ _____ ___ _______ 1977 11 1 2 1977 11 2 3 1977 12 1 4 1977 12 2 5 1978 1 1 6 1978 1 2 7
% Convert to datetime values
TT = timetable(datetime(T.Year,T.Month,T.Day),T.MaxTemp,'VariableNames',{'MaxTemp'})
TT = 6x1 timetable
Time MaxTemp ___________ _______ 01-Nov-1977 2 02-Nov-1977 3 01-Dec-1977 4 02-Dec-1977 5 01-Jan-1978 6 02-Jan-1978 7
% Get monthly max
TT_MonthlyMax = retime(TT,'Monthly','max')
TT_MonthlyMax = 3x1 timetable
Time MaxTemp ___________ _______ 01-Nov-1977 3 01-Dec-1977 5 01-Jan-1978 7
If any month is missing from the data, the MaxTemp for that month will be NaN. Those rows can be removed with,
TT_MonthlyMax(isnan(TT_MonthlyMax.MaxTemp),:) = [];

  20 Comments

Adam Danz
Adam Danz on 8 Dec 2020
Thanks for the explanation, Peter Perkins.
I appreciate that there's a difference between missing and empty values where missing numeric values are indicated with NaNs while empty numeric values are indicated with a 0x0 double array. An example that makes this distinction clear to other readers is data from a detector that samples at a fixed rate and unexpectedly skips a cycle (missing, NaN) vs a detector that works perfectly well but does not pick up on a signal (empty array). The missing and empty distinction is a necessity.
But there are two remaining problems IMO.
First is the inconsistency between this distinction in other data types. An empty cellstr is represented by {0x0 char} whereas a missing cellstr could be represented by {''} or perhaps {NaC} (Not-a-char, when the entire char array is 'missing') which would be consistent with cell arrays of doubles (NaN) or datetimes (NaT), etc. Outside of cell arrays, empty values aren't supported at all in double or char vectors:
x = [1 2 [] 4]
x = 1×3
1 2 4
s = ['a' 'b' '' 'd']
s = 'abd'
Second, it's not uncommon to treat empty values the same as missing values. For example, preallocation often uses cell(n,m) which creates a cell array of 0x0 doubles and some of those cells may remain empty due to missing data within the loop. It would be helpful if there were an option in the ismissing/rmmissing/fillmissing family of functions to include empties as missing which is currently the case only for cellstr data, to my knowledge. For example, F=fillmissing(A,method,emptyflag) similar to 'nanflag' in mean(),sum(),etc.
The problem is particularly common (and annoying) in tabular data with mixed classes.
Steven Lord
Steven Lord on 8 Dec 2020
An empty cellstr is by {0x0 char} whereas a missing cellstr could be represented by {''} or perhaps {NaC} (Not-a-char, when the entire char array is 'missing') which would be consistent with cell arrays of doubles (NaN) or datetimes (NaT), etc.
Instead of trying to find some way to represent missing data in a cellstr, I probably would store my text data (including missing) in a string array and use the string array instead of a cellstr.
A = ["apple"; "banana"; "cherry"; missing; "eggplant"]
A = 5×1 string array
"apple" "banana" "cherry" <missing> "eggplant"
Adam Danz
Adam Danz on 8 Dec 2020
Thanks, Steven Lord, strings do solve that problem and they distinguish between empties and missing.
A = ["apple"; ""; "cherry"; missing; "eggplant"]
A = 5×1 string array
"apple" "" "cherry" <missing> "eggplant"
But the bigger picture I'd like to emphasize is the option to treat empties as missing in the fillmissing(), rmmissing() etc family of functions. The current workaround is to find the empty arrays and replace them with missing values and for most data types, the missing() function comes in handy to fill with <missing> but missing() does not support cell arrays. So I'm suggesting that the _missing() family of functions optionally treat empty arrays as missing and to do this for empty cells as well.

Sign in to comment.

More Answers (0)

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!