Slope of some groups of data separated by NaNs

Hello there,
I have a dataset containing two variable, dx and z (please check attached file). The dx contain some groups of values separated by NaN. My intention is getting the slope line and its slope value for each group as shown below. Note that the dx values in each group contain/cross the negative and positive values. Thanks!
load('data_mine.mat');
figure;
scatter(dx,z);
set(gca, 'YDir','reverse')

 Accepted Answer

I am not certain what you want for ‘values’.
This prints the equation of the regression line for each line —
load('data_mine.mat')
whos('-file','data_mine.mat')
Name Size Bytes Class Attributes dx 1x893 7144 double z 1x893 7144 double
nanv = [0 ~isnan(dx) 0];
nanstart = strfind(nanv, [0 1]);
nanend = strfind(nanv, [1 0])-1;
nanmtx = [nanstart; nanend];
% [dxi,dx2] = bounds(dx)
[z1,z2] = bounds(z);
z1 = 5
z2 = 451
figure
scatter(dx,z)
set(gca, 'YDir','reverse')
hold on
for k = 1:size(nanmtx,2)
idxrng = nanmtx(1,k):nanmtx(2,k);
DM = [dx(idxrng); ones(size(idxrng))].';
B(:,k) = DM \ z(idxrng).';
x{k} = dx(idxrng);
y{k} = DM * B(:,k);
plot(x{k}, y{k}, '-g')
text(max(x{k}), max(y{k}), sprintf(' \\leftarrow z = %.3f \\cdot dx %+.3f', B(:,k)), 'Horiz','left', 'Vert','middle')
end
hold off
axis([-15 20 z1 z2])
xlabel('dx')
ylabel('z')
title('Entire Data Set')
B
B = 2×22
0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.4466 5.5000 11.2500 32.2500 35.5000 41.7500 106.0000 117.0000 127.7500 154.7500 270.2500 300.5000 330.7500 350.5000 359.5000 374.5000 389.2500 404.2500 410.0000 434.5000 437.2500 442.0000 448.6241
<mw-icon class=""></mw-icon>
<mw-icon class=""></mw-icon>
figure
scatter(dx,z)
set(gca, 'YDir','reverse')
hold on
for k = 1:size(nanmtx,2)
idxrng = nanmtx(1,k):nanmtx(2,k);
DM = [dx(idxrng); ones(size(idxrng))].';
% B(:,k) = DM \ z(idxrng).';
x{k} = dx(idxrng);
y{k} = DM * B(:,k);
plot(x{k}, y{k}, '-g')
text(max(x{k}), max(y{k}), sprintf(' \\leftarrow z = %.3f \\cdot dx %+.3f', B(:,k)), 'Horiz','left', 'Vert','middle')
end
hold off
axis([-15 20 100 135])
xlabel('dx')
ylabel('z')
title('Detail')
The regression coefficients are in the ‘B’ matrix, with the slope in the first row and intercept in tthe second row for each line.
.

15 Comments

Thank you @Star Strider. As most of the slope values quite equal, is it possible to somewhat "normalize" the slopes so that I can differ "the degree" of the differences?
My pleasure!
There are several ways to normalise them, and fortunately, MATLAB introduced the normalize functiion in R2018a to make this easier. Here, I chose the 'range' method to normalise them, that a scales them on a range of (0,1). (This seems to me to be the most appropriate, although other options are available. If you prefer a different method, I can change my code.)
Even this is not as straightforward as iit might first seem however, since almost all of them are so close to 1 that they only begin to differ in the 13th decimal place. So, in order to make that meaningful, I ‘stripped away’ the first 12 digits after the decimal, returning only the 13th and beyond with:
NT = rem((N*1E+12),1);
Here, ‘N’ is the normalised score returned by normalize. My code then presents the slopes, intercepts, and normalisation score (rename those as you wish) in the ‘Regression_Summary’ table. The table presents them in the order of their intercepts from lowest to highest, so they are the same as the order in the plot.
Try this —
load('data_mine.mat')
whos('-file','data_mine.mat')
Name Size Bytes Class Attributes dx 1x893 7144 double z 1x893 7144 double
nanv = [0 ~isnan(dx) 0];
nanstart = strfind(nanv, [0 1]);
nanend = strfind(nanv, [1 0])-1;
nanmtx = [nanstart; nanend];
% [dxi,dx2] = bounds(dx)
[z1,z2] = bounds(z);
figure
scatter(dx,z)
set(gca, 'YDir','reverse')
hold on
for k = 1:size(nanmtx,2)
idxrng = nanmtx(1,k):nanmtx(2,k);
DM = [dx(idxrng); ones(size(idxrng))].';
B(:,k) = DM \ z(idxrng).';
x{k} = dx(idxrng);
y{k} = DM * B(:,k);
plot(x{k}, y{k}, '-g')
text(max(x{k}), max(y{k}), sprintf(' \\leftarrow z = %.3f \\cdot dx %+.3f', B(:,k)), 'Horiz','left', 'Vert','middle')
end
hold off
axis([-15 20 z1 z2])
xlabel('dx')
ylabel('z')
title('Entire Data Set')
B
B = 2×22
0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.4466 5.5000 11.2500 32.2500 35.5000 41.7500 106.0000 117.0000 127.7500 154.7500 270.2500 300.5000 330.7500 350.5000 359.5000 374.5000 389.2500 404.2500 410.0000 434.5000 437.2500 442.0000 448.6241
<mw-icon class=""></mw-icon>
<mw-icon class=""></mw-icon>
[minslope,maxslope] = bounds(B(1,:))
minslope = 0.4466
maxslope = 0.5000
% format long
[N,C,S] = normalize(B(1,:),"range");
% disp(N)
NT = rem((N*1E+12),1);
Regression_Summary = array2table([B; NT].', 'VariableNames',{'Slope','Intercept','Score'});
disp(Regression_Summary)
Slope Intercept Score _______ _________ _______ 0.5 5.5 0.90845 0.5 11.25 0.90625 0.5 32.25 0.92297 0.5 35.5 0.8844 0.5 41.75 0.88867 0.5 106 0.9021 0.5 117 0.99988 0.5 127.75 0.8418 0.5 154.75 0.89587 0.5 270.25 0.83752 0.5 300.5 0.90002 0.5 330.75 0.92297 0.5 350.5 0.8584 0.5 359.5 0.9187 0.5 374.5 0.95203 0.5 389.25 0.69824 0.5 404.25 0.89587 0.5 410 0.9541 0.5 434.5 0.44019 0.5 437.25 0.55566 0.5 442 0.99988 0.44664 448.62 0
% figure
% scatter(dx,z)
% set(gca, 'YDir','reverse')
% hold on
% for k = 1:size(nanmtx,2)
% idxrng = nanmtx(1,k):nanmtx(2,k);
% DM = [dx(idxrng); ones(size(idxrng))].';
% % B(:,k) = DM \ z(idxrng).';
% x{k} = dx(idxrng);
% y{k} = DM * B(:,k);
% plot(x{k}, y{k}, '-g')
% text(max(x{k}), max(y{k}), sprintf(' \\leftarrow z = %.3f \\cdot dx %+.3f', B(:,k)), 'Horiz','left', 'Vert','middle')
% end
% hold off
% axis([-15 20 100 135])
% xlabel('dx')
% ylabel('z')
% title('Detail')
.
Great! Thank you @Star Strider.
Anyway, is the normalization you have suggested made by picking the maximum value of all slopes (~0.50000.....xx)? is it possible to set the normalization to a constant value, say 0.5 or 1 instead of using the maximum slope obtained from all slopes?
Since I have lots of profiles like this, so by setting like that, then I can use the scoring values comparable to my another profiles.
PS.
It seems that we need to modify this line
[N,C,S] = normalize(B(1,:),"range");
I don't know.
My pleasure!
Anyway, is the normalization you have suggested made by picking the maximum value of all slopes (~0.50000.....xx)?
The normalisation essentially ranks the slopes from 0 to 1 based on their relative values. All but one of them are tightly clustered in a relatively narrow range, so most of them are very close to each other. The only one that differs significantly from the others is the last one. (The table data are essentially rounded. Use the format function prior to creating the table to see all of the digits.)
‘... is it possible to set the normalization to a constant value, say 0.5 or 1 instead of using the maximum slope obtained from all slopes?
I am not certain what you want. It calculates the normalised scores based on all of the slopes given to it. It is possible to set a different range, and to use different methods to compute the normalisation. (See the normalize documentation section on methodtype for those options.) The one I chose, "range", seemed to me to be the most appropriate. I can probably do what you want if I understand what it is. Just now, I do not.
Can you provide an example? It does not have to be precise, or even use the actual values from these data. It just needs to be enough so that I understand what you want.
.
Thank you @Star Strider. Thanks for the explanation. Anyway, can we add additional info related to the center of depth (z) to the regression summary table? The center I meant is the center value of z for each slope. I really appreciate your help!
PS. The center of z must be located at x=0 of the slope I think.
Anyway @Star Strider, regarding the degree of difference of the slope I mentioned before, my concern is actually how to get the score or tightness score or degree of the difference of the slopes from a reference slope (for example slope 0.5) as shown in figure below. However, as the degree of the differences are very small, it's a bit tricky to handle that. But, your suggestion to consider only the last decimal places looks reasonable indeed.
PS.
My intention to compare the slopes with a reference slope (0.5) is my way to get a value which will be the degree standard for my another datasets. I don't know, it must be getting the ratio 0.5/slopes first I think before doing the treatments as your suggestion.
My pleasure!
The center I meant is the center value of z for each slope.
It would be straightforward to do that. One option is the median of the ‘y’-values, another is the mean of the first and last ‘y’-values I will add both, and you can choose the one you want.
You can experiment with those, however in the end I went with:
center_z(k) = mean(y{k}([1 end]));
because computing that wiith the actual ‘z’ values:
center_z(k) = mean(z(idxrng([1 end])));
produced results that were the same as the intercept.
‘... how to get the score or tightness score or degree of the difference of the slopes from a reference slope (for example slope 0.5) ...’
There are several possible ways to do that. One way would be to take the simple difference:
reference = 0.5;
dif_score = B(1,:) - reference;
the other would be to take the ‘relative difference’:
rel_score = (B(1,:) - reference) / reference;
I experimented wiith these and two different ways of scaling those as well (withn the array presented to each array2table function, so they do not appear as separate variables), those being:
dif_score/mean(dif_score)
and:
rel_score/median(rel_score)
It seems that:
dif_score/mean(dif_score)
and
rel_score/mean(rel_score)
produce the same result, as do:
dif_score/median(dif_score)
and
rel_score/median(rel_score)
however the results using the mean and median are much different, so I only showed one result for each, in different ‘Regression_Summary’ tables. (The mean is affected by extreme values in iits argument array, however the median is not.)
I can continue experimenting with diifferent options if you like, however I believe that these are reasonably close to the desired result.
Try these —
load('data_mine.mat')
whos('-file','data_mine.mat')
Name Size Bytes Class Attributes dx 1x893 7144 double z 1x893 7144 double
nanv = [0 ~isnan(dx) 0];
nanstart = strfind(nanv, [0 1]);
nanend = strfind(nanv, [1 0])-1;
nanmtx = [nanstart; nanend];
% [dxi,dx2] = bounds(dx)
[z1,z2] = bounds(z);
figure
scatter(dx,z)
set(gca, 'YDir','reverse')
hold on
for k = 1:size(nanmtx,2)
idxrng = nanmtx(1,k):nanmtx(2,k);
DM = [dx(idxrng); ones(size(idxrng))].';
B(:,k) = DM \ z(idxrng).';
x{k} = dx(idxrng);
y{k} = DM * B(:,k);
plot(x{k}, y{k}, '-g')
text(max(x{k}), max(y{k}), sprintf(' \\leftarrow z = %.3f \\cdot dx %+.3f', B(:,k)), 'Horiz','left', 'Vert','middle')
% center_z(k) = mean(z(idxrng([1 end])));
center_z(k) = mean(y{k}([1 end]));
end
hold off
axis([-15 20 z1 z2])
xlabel('dx')
ylabel('z')
title('Entire Data Set')
B
B = 2×22
0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.4466 5.5000 11.2500 32.2500 35.5000 41.7500 106.0000 117.0000 127.7500 154.7500 270.2500 300.5000 330.7500 350.5000 359.5000 374.5000 389.2500 404.2500 410.0000 434.5000 437.2500 442.0000 448.6241
<mw-icon class=""></mw-icon>
<mw-icon class=""></mw-icon>
[minslope,maxslope] = bounds(B(1,:))
minslope = 0.4466
maxslope = 0.5000
% format long
reference = 0.5;
dif_score = B(1,:) - reference;
rel_score = (B(1,:) - reference) / reference;
Regression_Summary = array2table([B; center_z; dif_score].', 'VariableNames',{'Slope','Intercept','Z Cennter','Score'});
disp(Regression_Summary)
Slope Intercept Z Cennter Score _______ _________ _________ ___________ 0.5 5.5 5.5 6.6613e-16 0.5 11.25 11.125 5.5511e-16 0.5 32.25 32.125 1.4433e-15 0.5 35.5 35.5 -6.1062e-16 0.5 41.75 41.125 -3.8858e-16 0.5 106 105.75 3.3307e-16 0.5 117 116.75 5.5511e-15 0.5 127.75 127.63 -2.8866e-15 0.5 154.75 154.87 0 0.5 270.25 270.12 -3.1086e-15 0.5 300.5 300.37 2.2204e-16 0.5 330.75 329.5 1.4433e-15 0.5 350.5 351 -1.9984e-15 0.5 359.5 359.38 1.2212e-15 0.5 374.5 375.25 2.9976e-15 0.5 389.25 389.5 -1.0547e-14 0.5 404.25 404.12 0 0.5 410 409.62 3.1086e-15 0.5 434.5 434.38 -2.4314e-14 0.5 437.25 437.38 -1.8152e-14 0.5 442 442.13 5.5511e-15 0.44664 448.62 448.18 -0.053362
Regression_Summary = array2table([B; center_z; rel_score].', 'VariableNames',{'Slope','Intercept','Z Cennter','Score'});
disp(Regression_Summary)
Slope Intercept Z Cennter Score _______ _________ _________ ___________ 0.5 5.5 5.5 1.3323e-15 0.5 11.25 11.125 1.1102e-15 0.5 32.25 32.125 2.8866e-15 0.5 35.5 35.5 -1.2212e-15 0.5 41.75 41.125 -7.7716e-16 0.5 106 105.75 6.6613e-16 0.5 117 116.75 1.1102e-14 0.5 127.75 127.63 -5.7732e-15 0.5 154.75 154.87 0 0.5 270.25 270.12 -6.2172e-15 0.5 300.5 300.37 4.4409e-16 0.5 330.75 329.5 2.8866e-15 0.5 350.5 351 -3.9968e-15 0.5 359.5 359.38 2.4425e-15 0.5 374.5 375.25 5.9952e-15 0.5 389.25 389.5 -2.1094e-14 0.5 404.25 404.12 0 0.5 410 409.62 6.2172e-15 0.5 434.5 434.38 -4.8628e-14 0.5 437.25 437.38 -3.6304e-14 0.5 442 442.13 1.1102e-14 0.44664 448.62 448.18 -0.10672
Regression_Summary = array2table([B; center_z; dif_score/mean(dif_score)].', 'VariableNames',{'Slope','Intercept','Z Cennter','Score'});
disp(Regression_Summary)
Slope Intercept Z Cennter Score _______ _________ _________ ___________ 0.5 5.5 5.5 -2.7463e-13 0.5 11.25 11.125 -2.2886e-13 0.5 32.25 32.125 -5.9504e-13 0.5 35.5 35.5 2.5175e-13 0.5 41.75 41.125 1.602e-13 0.5 106 105.75 -1.3732e-13 0.5 117 116.75 -2.2886e-12 0.5 127.75 127.63 1.1901e-12 0.5 154.75 154.87 0 0.5 270.25 270.12 1.2816e-12 0.5 300.5 300.37 -9.1545e-14 0.5 330.75 329.5 -5.9504e-13 0.5 350.5 351 8.239e-13 0.5 359.5 359.38 -5.035e-13 0.5 374.5 375.25 -1.2359e-12 0.5 389.25 389.5 4.3484e-12 0.5 404.25 404.12 0 0.5 410 409.62 -1.2816e-12 0.5 434.5 434.38 1.0024e-11 0.5 437.25 437.38 7.4838e-12 0.5 442 442.13 -2.2886e-12 0.44664 448.62 448.18 22
Regression_Summary = array2table([B; center_z; rel_score/median(rel_score)].', 'VariableNames',{'Slope','Intercept','Z Cennter','Score'});
disp(Regression_Summary)
Slope Intercept Z Cennter Score _______ _________ _________ ___________ 0.5 5.5 5.5 6 0.5 11.25 11.125 5 0.5 32.25 32.125 13 0.5 35.5 35.5 -5.5 0.5 41.75 41.125 -3.5 0.5 106 105.75 3 0.5 117 116.75 50 0.5 127.75 127.63 -26 0.5 154.75 154.87 0 0.5 270.25 270.12 -28 0.5 300.5 300.37 2 0.5 330.75 329.5 13 0.5 350.5 351 -18 0.5 359.5 359.38 11 0.5 374.5 375.25 27 0.5 389.25 389.5 -95 0.5 404.25 404.12 0 0.5 410 409.62 28 0.5 434.5 434.38 -219 0.5 437.25 437.38 -163.5 0.5 442 442.13 50 0.44664 448.62 448.18 -4.8064e+14
.
Perfect! Many thanks @Star Strider! My last question for this thread (I hope), is there any suggestion to create plot the Score (I prefer the median-base)? I know, the last value (-4e+14) will "annoy" the representation of the plot, but let me know if there any plot suggestion.
Let's say, we ignore such extreme value, then create a bar graph for z_center (y axis) vs rel_score/median(rel_score) as the x-axis by straightforward using:
bar(rel_score/median(rel_score),center_z)
I got this message: XData values must be unique
Any suggestion?
As always, my pleasure!
I berlieeve that you want the arguments to bar in the reverse order. That is probably the problem.
Omitting the extreme value, the bar chart of the scaled scores is straightforward, and a hiistogram is also straightforward —
load('data_mine.mat')
whos('-file','data_mine.mat')
Name Size Bytes Class Attributes dx 1x893 7144 double z 1x893 7144 double
nanv = [0 ~isnan(dx) 0];
nanstart = strfind(nanv, [0 1]);
nanend = strfind(nanv, [1 0])-1;
nanmtx = [nanstart; nanend];
% [dxi,dx2] = bounds(dx)
[z1,z2] = bounds(z);
figure
scatter(dx,z)
set(gca, 'YDir','reverse')
hold on
for k = 1:size(nanmtx,2)
idxrng = nanmtx(1,k):nanmtx(2,k);
DM = [dx(idxrng); ones(size(idxrng))].';
B(:,k) = DM \ z(idxrng).';
x{k} = dx(idxrng);
y{k} = DM * B(:,k);
plot(x{k}, y{k}, '-g')
text(max(x{k}), max(y{k}), sprintf(' \\leftarrow z = %.3f \\cdot dx %+.3f', B(:,k)), 'Horiz','left', 'Vert','middle')
% center_z(k) = mean(z(idxrng([1 end])));
center_z(k) = mean(y{k}([1 end]));
end
hold off
axis([-15 20 z1 z2])
xlabel('dx')
ylabel('z')
title('Entire Data Set')
B
B = 2×22
0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.4466 5.5000 11.2500 32.2500 35.5000 41.7500 106.0000 117.0000 127.7500 154.7500 270.2500 300.5000 330.7500 350.5000 359.5000 374.5000 389.2500 404.2500 410.0000 434.5000 437.2500 442.0000 448.6241
<mw-icon class=""></mw-icon>
<mw-icon class=""></mw-icon>
[minslope,maxslope] = bounds(B(1,:))
minslope = 0.4466
maxslope = 0.5000
% format long
reference = 0.5;
dif_score = B(1,:) - reference;
dif_score_scaled = dif_score/median(dif_score);
rel_score = (B(1,:) - reference) / reference;
rel_score_scaled = rel_score/median(rel_score);
Regression_Summary = array2table([B; center_z; dif_score_scaled].', 'VariableNames',{'Slope','Intercept','Z Cennter','Difference Score (Scaled)'});
disp(Regression_Summary)
Slope Intercept Z Cennter Difference Score (Scaled) _______ _________ _________ _________________________ 0.5 5.5 5.5 6 0.5 11.25 11.125 5 0.5 32.25 32.125 13 0.5 35.5 35.5 -5.5 0.5 41.75 41.125 -3.5 0.5 106 105.75 3 0.5 117 116.75 50 0.5 127.75 127.63 -26 0.5 154.75 154.87 0 0.5 270.25 270.12 -28 0.5 300.5 300.37 2 0.5 330.75 329.5 13 0.5 350.5 351 -18 0.5 359.5 359.38 11 0.5 374.5 375.25 27 0.5 389.25 389.5 -95 0.5 404.25 404.12 0 0.5 410 409.62 28 0.5 434.5 434.38 -219 0.5 437.25 437.38 -163.5 0.5 442 442.13 50 0.44664 448.62 448.18 -4.8064e+14
Regression_Summary = array2table([B; center_z; rel_score_scaled].', 'VariableNames',{'Slope','Intercept','Z Cennter','Relative Score (Scaled)'});
disp(Regression_Summary)
Slope Intercept Z Cennter Relative Score (Scaled) _______ _________ _________ _______________________ 0.5 5.5 5.5 6 0.5 11.25 11.125 5 0.5 32.25 32.125 13 0.5 35.5 35.5 -5.5 0.5 41.75 41.125 -3.5 0.5 106 105.75 3 0.5 117 116.75 50 0.5 127.75 127.63 -26 0.5 154.75 154.87 0 0.5 270.25 270.12 -28 0.5 300.5 300.37 2 0.5 330.75 329.5 13 0.5 350.5 351 -18 0.5 359.5 359.38 11 0.5 374.5 375.25 27 0.5 389.25 389.5 -95 0.5 404.25 404.12 0 0.5 410 409.62 28 0.5 434.5 434.38 -219 0.5 437.25 437.38 -163.5 0.5 442 442.13 50 0.44664 448.62 448.18 -4.8064e+14
Lv_dif_score_scaled = ~isoutlier(dif_score_scaled);
Lv_rel_score_scaled = ~isoutlier(rel_score_scaled);
dif_score_scaled = dif_score_scaled(Lv_dif_score_scaled);
rel_score_scaled = rel_score_scaled(Lv_rel_score_scaled);
center_z = center_z(Lv_dif_score_scaled);
[Nds, Eds] = histcounts(dif_score_scaled, 11);
xds = Eds(1:end-1) + diff(Eds)/2;
[Nrs, Ers] = histcounts(rel_score_scaled, 11);
xrs = Ers(1:end-1) + diff(Ers)/2;
figure
bar(center_z, dif_score_scaled)
grid
axis('padded')
xlabel('Center Z')
ylabel('Score')
title('Difference Score (Scaled)')
figure
bar(xds, Nds)
grid
ylim('padded')
xlabel('Score')
ylabel('Number')
title('Difference Score (Scaled)')
figure
bar(center_z, rel_score_scaled)
grid
axis('padded')
xlabel('Center Z')
ylabel('Score')
title('Relative Score (Scaled)')
figure
bar(xrs, Nrs)
grid
ylim('padded')
xlabel('Score')
ylabel('Number')
title('Relative Score (Scaled)')
The ‘Lv_’ vectors are logical vectors returned by isoutlier that detect the outliers. Here, I have them return only the values that are not outliers, and use that to select the data that are not outliers (using the ~ ‘logical not’.operator).
I added the histograms out of my own interest, to see how the data are distributed.
.
Nice! sorry @Star Strider, my extra question then, is it possible to modify the bar plot flipped right (the Center z as y-axis)? so that I can subplot them aside with the slopes plot. It seems that it can't be done straightforward by changing the order of the bar syntax bar(rel_score_scaled, center_z). Sorry, I'm not too familiar yet with bar plot behaviour.
The barh function should do that. Change the bar calls to barh. No other thanges to my code will be necessary, although you can add:
Ax = gca;
Ax.YDir = 'reverse';
to make them compatible with the other plot. (I did that here, delete those liines if you don’t want them.)
load('data_mine.mat')
whos('-file','data_mine.mat')
Name Size Bytes Class Attributes dx 1x893 7144 double z 1x893 7144 double
nanv = [0 ~isnan(dx) 0];
nanstart = strfind(nanv, [0 1]);
nanend = strfind(nanv, [1 0])-1;
nanmtx = [nanstart; nanend];
% [dxi,dx2] = bounds(dx)
[z1,z2] = bounds(z);
figure
scatter(dx,z)
set(gca, 'YDir','reverse')
hold on
for k = 1:size(nanmtx,2)
idxrng = nanmtx(1,k):nanmtx(2,k);
DM = [dx(idxrng); ones(size(idxrng))].';
B(:,k) = DM \ z(idxrng).';
x{k} = dx(idxrng);
y{k} = DM * B(:,k);
plot(x{k}, y{k}, '-g')
text(max(x{k}), max(y{k}), sprintf(' \\leftarrow z = %.3f \\cdot dx %+.3f', B(:,k)), 'Horiz','left', 'Vert','middle')
% center_z(k) = mean(z(idxrng([1 end])));
center_z(k) = mean(y{k}([1 end]));
end
hold off
axis([-15 20 z1 z2])
xlabel('dx')
ylabel('z')
title('Entire Data Set')
B
B = 2×22
0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.4466 5.5000 11.2500 32.2500 35.5000 41.7500 106.0000 117.0000 127.7500 154.7500 270.2500 300.5000 330.7500 350.5000 359.5000 374.5000 389.2500 404.2500 410.0000 434.5000 437.2500 442.0000 448.6241
<mw-icon class=""></mw-icon>
<mw-icon class=""></mw-icon>
[minslope,maxslope] = bounds(B(1,:))
minslope = 0.4466
maxslope = 0.5000
% format long
reference = 0.5;
dif_score = B(1,:) - reference;
dif_score_scaled = dif_score/median(dif_score);
rel_score = (B(1,:) - reference) / reference;
rel_score_scaled = rel_score/median(rel_score);
Regression_Summary = array2table([B; center_z; dif_score_scaled].', 'VariableNames',{'Slope','Intercept','Z Cennter','Difference Score (Scaled)'});
disp(Regression_Summary)
Slope Intercept Z Cennter Difference Score (Scaled) _______ _________ _________ _________________________ 0.5 5.5 5.5 6 0.5 11.25 11.125 5 0.5 32.25 32.125 13 0.5 35.5 35.5 -5.5 0.5 41.75 41.125 -3.5 0.5 106 105.75 3 0.5 117 116.75 50 0.5 127.75 127.63 -26 0.5 154.75 154.87 0 0.5 270.25 270.12 -28 0.5 300.5 300.37 2 0.5 330.75 329.5 13 0.5 350.5 351 -18 0.5 359.5 359.38 11 0.5 374.5 375.25 27 0.5 389.25 389.5 -95 0.5 404.25 404.12 0 0.5 410 409.62 28 0.5 434.5 434.38 -219 0.5 437.25 437.38 -163.5 0.5 442 442.13 50 0.44664 448.62 448.18 -4.8064e+14
Regression_Summary = array2table([B; center_z; rel_score_scaled].', 'VariableNames',{'Slope','Intercept','Z Cennter','Relative Score (Scaled)'});
disp(Regression_Summary)
Slope Intercept Z Cennter Relative Score (Scaled) _______ _________ _________ _______________________ 0.5 5.5 5.5 6 0.5 11.25 11.125 5 0.5 32.25 32.125 13 0.5 35.5 35.5 -5.5 0.5 41.75 41.125 -3.5 0.5 106 105.75 3 0.5 117 116.75 50 0.5 127.75 127.63 -26 0.5 154.75 154.87 0 0.5 270.25 270.12 -28 0.5 300.5 300.37 2 0.5 330.75 329.5 13 0.5 350.5 351 -18 0.5 359.5 359.38 11 0.5 374.5 375.25 27 0.5 389.25 389.5 -95 0.5 404.25 404.12 0 0.5 410 409.62 28 0.5 434.5 434.38 -219 0.5 437.25 437.38 -163.5 0.5 442 442.13 50 0.44664 448.62 448.18 -4.8064e+14
Lv_dif_score_scaled = ~isoutlier(dif_score_scaled);
Lv_rel_score_scaled = ~isoutlier(rel_score_scaled);
dif_score_scaled = dif_score_scaled(Lv_dif_score_scaled);
rel_score_scaled = rel_score_scaled(Lv_rel_score_scaled);
center_z = center_z(Lv_dif_score_scaled);
[Nds, Eds] = histcounts(dif_score_scaled, 11);
xds = Eds(1:end-1) + diff(Eds)/2;
[Nrs, Ers] = histcounts(rel_score_scaled, 11);
xrs = Ers(1:end-1) + diff(Ers)/2;
figure
barh(center_z, dif_score_scaled)
grid
axis('padded')
xlabel('Center Z')
ylabel('Score')
title('Difference Score (Scaled)')
Ax = gca;
Ax.YDir = 'reverse';
figure
barh(xds, Nds)
grid
ylim('padded')
xlabel('Score')
ylabel('Number')
title('Difference Score (Scaled)')
figure
barh(center_z, rel_score_scaled)
grid
axis('padded')
xlabel('Center Z')
ylabel('Score')
title('Relative Score (Scaled)')
Ax = gca;
Ax.YDir = 'reverse';
figure
barh(xrs, Nrs)
grid
ylim('padded')
xlabel('Score')
ylabel('Number')
title('Relative Score (Scaled)')
.
Great! Thank you very much! Sorry, my last last question @Star Strider. I just test with my other datasets (see my new attached file). I found the score value is infinite (inf). Is there something wrong with the code or it should be "0" so I can simply replace it with "0" as the score of the slope?
As always, my pleasure!
I had not anticipated that the median of ‘dif_score’ or ‘rel_score’ could be 0. I changed that section of my code to:
mds = median(dif_score)+eps;
dif_score_scaled = dif_score/mds;
rel_score = (B(1,:) - reference) / reference;
mrs = median(rel_score)+eps;
rel_score_scaled = rel_score/mrs;
to eliminate that possibility, while not significantly altering the results of the other calculations.
It now returns 0 raather than Inf or NaN when the numerator of those calculations is 0.
Also, in order for the 0 values to show up on the barh plots, I changed the BaseLine property to:
bh2.BaseLine.LineStyle = ":";
Now, they can at least be seen.
Try this —
load('data_mine_3rd.mat')
whos('-file','data_mine_3rd.mat')
Name Size Bytes Class Attributes dx 1x997 7976 double z 1x997 7976 double
nanv = [0 ~isnan(dx) 0];
nanstart = strfind(nanv, [0 1]);
nanend = strfind(nanv, [1 0])-1;
nanmtx = [nanstart; nanend];
% [dxi,dx2] = bounds(dx)
[z1,z2] = bounds(z);
figure
scatter(dx,z)
set(gca, 'YDir','reverse')
hold on
for k = 1:size(nanmtx,2)
idxrng = nanmtx(1,k):nanmtx(2,k);
DM = [dx(idxrng); ones(size(idxrng))].';
B(:,k) = DM \ z(idxrng).';
x{k} = dx(idxrng);
y{k} = DM * B(:,k);
plot(x{k}, y{k}, '-g')
text(max(x{k}), max(y{k}), sprintf(' \\leftarrow z = %.3f \\cdot dx %+.3f', B(:,k)), 'Horiz','left', 'Vert','middle')
% center_z(k) = mean(z(idxrng([1 end])));
center_z(k) = mean(y{k}([1 end]));
end
hold off
axis([-15 20 z1 z2])
xlabel('dx')
ylabel('z')
title('Entire Data Set')
B
B = 2×9
0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 0.5000 15.5000 29.0000 57.5000 90.5000 158.5000 497.0000 529.5000 694.5000 773.0000
<mw-icon class=""></mw-icon>
<mw-icon class=""></mw-icon>
[minslope,maxslope] = bounds(B(1,:))
minslope = 0.5000
maxslope = 0.5000
% format long
reference = 0.5;
dif_score = B(1,:) - reference;
mds = median(dif_score)+eps;
dif_score_scaled = dif_score/mds;
rel_score = (B(1,:) - reference) / reference;
mrs = median(rel_score)+eps;
rel_score_scaled = rel_score/mrs;
Regression_Summary = array2table([B; center_z; dif_score_scaled].', 'VariableNames',{'Slope','Intercept','Z Cennter','Difference Score (Scaled)'});
disp(Regression_Summary)
Slope Intercept Z Cennter Difference Score (Scaled) _____ _________ _________ _________________________ 0.5 15.5 15 -1.75 0.5 29 29 2 0.5 57.5 57.5 0 0.5 90.5 90.25 7.5 0.5 158.5 158.5 0 0.5 497 496.25 -33.5 0.5 529.5 529.5 0 0.5 694.5 695 -12.75 0.5 773 772.75 34.5
Regression_Summary = array2table([B; center_z; rel_score_scaled].', 'VariableNames',{'Slope','Intercept','Z Cennter','Relative Score (Scaled)'});
disp(Regression_Summary)
Slope Intercept Z Cennter Relative Score (Scaled) _____ _________ _________ _______________________ 0.5 15.5 15 -3.5 0.5 29 29 4 0.5 57.5 57.5 0 0.5 90.5 90.25 15 0.5 158.5 158.5 0 0.5 497 496.25 -67 0.5 529.5 529.5 0 0.5 694.5 695 -25.5 0.5 773 772.75 69
Lv_dif_score_scaled = ~isoutlier(dif_score_scaled);
Lv_rel_score_scaled = ~isoutlier(rel_score_scaled);
dif_score_scaled = dif_score_scaled(Lv_dif_score_scaled);
rel_score_scaled = rel_score_scaled(Lv_rel_score_scaled);
center_z = center_z(Lv_dif_score_scaled);
[Nds, Eds] = histcounts(dif_score_scaled, 11);
xds = Eds(1:end-1) + diff(Eds)/2;
[Nrs, Ers] = histcounts(rel_score_scaled, 11);
xrs = Ers(1:end-1) + diff(Ers)/2;
figure
bh1 = barh(center_z, dif_score_scaled);
bh1.BaseLine.LineStyle = ":";
grid
axis('padded')
xlabel('Center Z')
ylabel('Score')
title('Difference Score (Scaled)')
Ax = gca;
Ax.YDir = 'reverse';
figure
barh(xds, Nds)
grid
ylim('padded')
xlabel('Score')
ylabel('Number')
title('Difference Score (Scaled)')
figure
bh2 = barh(center_z, rel_score_scaled);
grid
bh2.BaseLine.LineStyle = ":";
axis('padded')
xlabel('Center Z')
ylabel('Score')
title('Relative Score (Scaled)')
Ax = gca;
Ax.YDir = 'reverse';
figure
barh(xrs, Nrs)
grid
ylim('padded')
xlabel('Score')
ylabel('Number')
title('Relative Score (Scaled)')
.
Perfect! Thank you very much!
As always, my pleasure!
This was an interesting problem!

Sign in to comment.

More Answers (1)

Hi @Adi Purwandana,

After reviewing your comments, and analyzing the dataset data_mine.mat in matlab shown below

Name      Size             Bytes   Class     Attributes
dx        1x893            7144   double              
z         1x893            7144   double   

You are right that it contains two variables, dx and z. As you mentioned that dx having groups of values separated by NaNs, first import the .mat. file to access the variables as shown in your code snippet. Then, split the dx values into separate groups based on the NaN values. For each group, fit a linear regression model to determine the slope and then visualize each group along with its corresponding slope line. Here is a full code example in MATLAB that accomplishes this:

% Load the dataset
load('data_mine.mat');
% Initialize variables
dx_groups = {};
z_groups = {};
current_group_dx = [];
current_group_z = [];
% Split dx and z into groups based on NaN in dx
for i = 1:length(dx)
  if isnan(dx(i))
      if ~isempty(current_group_dx) % Only save if current group is not empty
          dx_groups{end + 1} = current_group_dx; %#ok<AGROW>
          z_groups{end + 1} = current_group_z; %#ok<AGROW>
          current_group_dx = []; % Reset for next group
          current_group_z = []; % Reset for next group
      end
    else
      current_group_dx(end + 1) = dx(i); %#ok<AGROW>
      current_group_z(end + 1) = z(i); %#ok<AGROW>
  end
end
% Add last group if it exists
if ~isempty(current_group_dx)
  dx_groups{end + 1} = current_group_dx; %#ok<AGROW>
  z_groups{end + 1} = current_group_z; %#ok<AGROW>
end
% Initialize figure for plotting
figure;
hold on;
% Analyze each group and plot results
for g = 1:length(dx_groups)
  % Extract current group
  x = dx_groups{g};
  y = z_groups{g};
    % Perform linear regression to find slope
    p = polyfit(x, y, 1); % p(1) is the slope, p(2) is the intercept
    % Generate x values for plotting the fitted line
    x_fit = linspace(min(x), max(x), 100);
    y_fit = polyval(p, x_fit);
    % Plot original data points
    plot(x, y, 'o', 'DisplayName', ['Group ' num2str(g)]);
    % Plot the fitted line
    plot(x_fit, y_fit, 'LineWidth', 2, 'DisplayName', ['Slope: ' num2str(p(1))]);
  end
% Finalize plot settings
xlabel('dx');
ylabel('z');
title('Slope Analysis of Groups in dx');
legend show;
grid on;
hold off;
% Display slopes for each group in command window
for g = 1:length(dx_groups)
  fprintf('Group %d: Slope = %.4f\n', g, polyfit(dx_groups{g}, z_groups{g}, 
  1));
end

Please see attached.

As you will notice that the code starts by loading the data_mine.mat file.It loops through dx, collecting values until a NaN is encountered. Each segment is stored in separate cell arrays (dx_groups and z_groups). For each group of data points, a linear fit is performed using polyfit, which returns coefficients for a linear equation (y = mx + b). For more information on this function, please refer to polyfit

Each group’s data points and corresponding fitted line are plotted. The slopes are displayed in the legend and printed to the console. Adjust visualization parameters (like colors or markers) based on your preferences or specific requirements.

Hope this helps.

Please let me know if you have any further questions.

Products

Release

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!