Is there a way to average sequential data on an array and then be able to split the array?

I want to be able to split an array of n rows and 5 columns into sections every time the data of a specific column (let's call it column3) crosses zero, and every time there is a peak or valley. I want the split of the array to happen to all the data (the conditions are based on column3, but are applied to all the columns).

The data also has repeated values for the whole data set; for those values I want to average the data of each column by considering the data that was averaged for a specific column per section.

if true
%code example
    column3 = [-6 -6 -6 -6 -5 -5 -3 -3 -3 -1 -1 -1 0 0 2 2 2 3 3 5 5 5 5 6 6 6 6 4 4 4 3 3 3 3 3 3 3 2 2 2 1 1 1 1 0 0 -1 -1 -2 -3 -5 -5 -5 -5 -5 -5 -5 -6 -6 -6];

end

I want this to end up looking like

if true
  %code result
column3_1 = [-6 -5 -3 -1 0]
column3_2 = [0 2 3 5 6]
column3_3 = [6 4 3 2 1 0]
column3_4 = [0 -1 -5 -3 -5 -6]
end

Then these changes should be also applied to the other columns based on the rows that were averaged for column3.

-----

My line of thought is first I must apply a code to average all the values that are repeated and in sequence using a loop, then separate the sections based on peaks, valleys, and crossing zeroes.

if true
  % code thought process
%first average all values
column3_avg = [-6 -5 -3 -1 -0 2 3 5 6 4 3 2 1 0 -1 -2 -3 -5 -6]
%then divide into sections
column3_1 = [-6 -5 -3 -1 0]
column3_2 = [0 2 3 5 6]
column3_3 = [6 4 3 2 1 0]
column3_4 = [0 -1 -5 -3 -5 -6]

end

The main issue is that I can't seem to average only equal sequential data. Is there an effective way to do this on Matlab with a loop or am I better off doing it all by hand?

Thank you in advance

Answer 1

[Guillaume]

The modern and easy way to do what you want is certainly not by hand, and not with a loop either:

demomatrix = rand(60, 10);
demomatrix(:, 3) =  [-6 -6 -6 -6 -5 -5 -3 -3 -3 -1 -1 -1 0 0 2 2 2 3 3 5 5 5 5 6 6 6 6 4 4 4 3 3 3 3 3 3 3 2 2 2 1 1 1 1 0 0 -1 -1 -2 -3 -5 -5 -5 -5 -5 -5 -5 -6 -6 -6];

%averaging of identical consecutive values:
groupids = cumsum([1; diff(demomatrix(:, 3))~=0]);
averagedmatrix = splitapply(@(rows) mean(rows, 1), demomatrix, groupids)

%splitting at 0 and direction reversal:
groupids = cumsum([false; diff(sign(diff(averagedmatrix(:, 3))))~=0; false] ...identify change of direction
                  | averagedmatrix(:, 3) == 0) + 1;                           %identify zeros    
splitaverage = splitapply(@(rows) {rows}, averagedmatrix, groupids);

Now, you want the transition point to belong to both side. A bit unusual, but can be adjusted after the fact, with a loop indeed:

for row = 1:numel(splitaverage)-1
   splitaverage{row}(end+1, :) = splitaverage{row+1}(1, :);
end
celldisp(splitaverage)

Note that the result is stored in a cell array. Do not create separate variables for it, use indexing. Instead of your colum3_4, use

splitaverage{4}

[yz]

Thank you so much.

Answer 2

[Aveek Podder]

Hopefully this solves the problem. The Cell type variable ‘sections’ contains the split data and ‘column3’ contains the column data.

if true
  % code
  column3 = [-6 -6 -6 -6 -5 -5 -3 -3 -3 -1 -1 -1 0 0 2 2 2 3 3 5 5 5 5 6 ...
      6 6 6 4 4 4 3 3 3 3 3 3 3 2 2 2 1 1 1 1 0 0 -1 -1 -2 -3 -5 -5 -5 ...
      -5 -5 -5 -5 -6 -6 -6];

    % First the Average part is done
    column3_avg = column3([1,diff(column3)]~=0);

    % Dividing it into Sections
    diff_col = [0,diff(column3_avg)];
    strt_pnt = 1;
    sections = cell(0);
    for i = 1:length(column3_avg)
        end_pnt = i;

        if i == length(column3_avg)
            sections{length(sections)+1} = column3_avg(strt_pnt:end_pnt);
            break;
        end

        if column3_avg(i) == 0 || (diff_col(i)*diff_col(i+1) < 0)
            sections{length(sections)+1} = column3_avg(strt_pnt:end_pnt);
            strt_pnt = i;
        end
    end

    % Accessing the data
    num_of_sections = length(sections)
    column3_1 = sections{1}
    column3_2 = sections{2}
    column3_4 = sections{4}

end

[John BG]

Hi

this is John BG <mailto:[e-mail] [e-mail]>

I have accepted Aveek Poder's answer because it meets all questions requirements, while Guillaume's although solving the question, tells the question originator not to use the set of variables that have been precisely detailed to collect the results.

I would recommend the use of command evalin to dynamically generate variables

column3_XX

in case the input sequence requires many more output variables.

[Stephen23]

"I would recommend the use of command evalin to dynamically generate variables"

Really? Why would you recommend to other beginners that they should write slow, buggy, hard-to-debug, obfuscated code that is inefficient, causes so many problems, and is a total waste of time for accessing variables in a loop? Why would you recommend exactly the opposite of what the MATLAB documentation recommends: "Avoid functions such as eval, evalc, evalin, and feval(fname).". How did you end up knowing more about what to recommend than the people who wrote the MATLAB documentation?

Is explaining how JIT works is a bit too complicated for some beginners, or do they simply not want to make an effort learning?

https://www.mathworks.com/matlabcentral/answers/352237-is-there-the-more-elegant-way-to-do-this

[John BG]

well, now that you ask, it's good to know what's in the arsenal, so in case you need it you know it's there.

Precisely because as you mention it, YZ may be a beginner, may be not, but if so, if never acquainted earlier on with the powerful command evalin, why not mentioning evalin.

Now YZ knows evalin exists, and then YZ chooses, decides, makes a selection, free choice, unconditioned sorting among the available possibilities. Not you, not me, but the question originator: YZ.

Helping other is about giving them, we the answerers give them, the questioners.

If you keep beginners down to level 1, the chances are that they will never learn how to use sharper tools.

If they crash their debugger, it's not your debugger, is it? it's not my debugger, it is not. We humans learn by doing mistakes.

If you show the whole contents of the toolbox, then they are free to decide:

It may read Biblical if not a believer, but I find the following New Testament line appropriate:

Know the Truth and (only then) You are free (from the obscurity that represents not knowing the existence of all possibilities)

regards

John BG

<mailto:[e-mail] [e-mail]>

[Guillaume]

@John, I personally, find it totally unnaccepatable to go around and accept answers on behalf of other people.

particularly, for a topic in which you were not involved and for such a bad reason. There has been many conversations now where you have been told that telling beginners to use eval and co. is extremely bad advice.

You did not even give a chance to the original poster to make a choice as to which answer they prefer. Both answers are only a few hours old.

I will be asking Mathworks to revert your acceptation.

[Stephen23]

"I will be asking Mathworks to revert your acceptation."

The answer itself is quite okay. Accepting it just gave points to Aveek Podder.

[Guillaume]

The harm is that John usurped the authority of the original questioner. It is up to the original poster to decide which solution is best (I happen to think mine is better).

Certainly, not John, and certainly not for the reason stated.

The reputation points are irrelevant. The principle is.

As as been discussed before, accepting an answer does not just give reputation points, it also puts the accepted answer in a lot more prominence.

[yz]

Thank you Guillaume. I feel your answer better suits my original question. I highly appreciate the addition of the other columns so I could better understand what is happening with the code.

I do not appreciate someone accepting an answer on my behalf, especially when I hadn't even had a chance to look at the answers. 7 hours ago I was asleep. While I appreciate John's readiness to evaluate the answers, I feel the poster should be given at least a day to compare answers and see how well they work for their purposes.

I do want to thank both Aveek and Guillaume for answering. I do really appreciate it.

Thank you,

- yz