Identifying significantly different pairs from multcompare output

3 views (last 30 days)
edward holt
edward holt on 30 Jan 2020
Edited: Scott MacKenzie on 31 Jul 2021
Data attached.
Using R2019a
I am trying to identify from the 'c' output of multcompare the significantly different pairs (i.e. without having to use the interactive figure)
In this example the confidence interval does not contain 0, so the difference is significant at the 5% significance level. If the confidence interval did contain 0, the difference would not be significant
This suggests that significantly different pairs are not the rows in c where the lower bound (column 3) < 0 & the upper bound (column 5) > 0.
However in the attached example below this doesn't quite appear to be the case.
load exampleinput.mat
[p, tbl, stats] = anova1(exampleinput.D,exampleinput.form,"ON");
[c,m,~,gnames] = multcompare(stats,"Alpha",0.05,"CType","bonferroni")
significantpairs = c(~(c(:,3) < 0 & c(:,5) > 0),:)
pairs = [gnames(significantpairs(:,1)), gnames(significantpairs(:,2))]
The above code nearly gets it right.
the interactive graph identifies FCK and WBCK as being different, the code does not.
The code identifies FCK and SECK as being different, the graph does not.
The 5 other pairs are correctly identified.
I have tried to extract the different pairs by writing a function utilising this method, as suggested here. This didn't quite work.
Changing CType to 'tukey-kramer' (suggested here by Peter König), yields similiar results, some pairs are correctly identified, some are not.
So, the question is: which to trust: the interactive graph, or the code? Or rather, what is the interactive graph doing that I'm not?
Thank you.

Answers (1)

Scott MacKenzie
Scott MacKenzie on 30 Jul 2021
Edited: Scott MacKenzie on 31 Jul 2021
The significantly different pairs are identified by the p-values in column 6 of the c matrix generated by multcompare. Here's an example using some test data (attached). There are four columns of data representing marks on four tests. The researcher wishes to know which test pairs have marks that significantly differ from each other.
% load test data: 16 marks on 4 tests
y = readmatrix('testdata.txt'); % 16x4
% do the anova
[~, ~, stats] = anova1(y, {'t1' 't2' 't3' 't4'}, 'off');
% pass stats object into multcompare
[c, ~, ~, ~] = multcompare(stats, 'display', 'off')
c = 6×6
1.0000 2.0000 -3.9088 -0.8750 2.1588 0.8711 1.0000 3.0000 -7.5338 -4.5000 -1.4662 0.0013 1.0000 4.0000 -4.8463 -1.8125 1.2213 0.3983 2.0000 3.0000 -6.6588 -3.6250 -0.5912 0.0130 2.0000 4.0000 -3.9713 -0.9375 2.0963 0.8464 3.0000 4.0000 -0.3463 2.6875 5.7213 0.1002
The summary results (using the default alpha of .05) are in the 6th column above. The marks are significantly different between test 1 and test 3 (p = .0013) and between test 2 and test 3 (p = .0130).
The default test is Tukey-Kramer. A different test can be selected via the ctype option. Generally, the tests yield the same or similar results.
NOTE: There is more than one version of multcompare. The output is a bit different for the version of multcompare that receives a repeated measures model as input, rather than a stats object (as demonstrated above). Check the documentation for details




Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!