MATLAB Answers

Which pairwise hypothesis test is used on multcompare

23 views (last 30 days)
Reinder Dorman
Reinder Dorman on 3 Dec 2020
Commented: Hamed Nassar on 7 Nov 2021 at 19:03
HI All,
I have a feeling this question should be easily answered, so I suspect I might just be missing something.
On the function page of multcompare it aptly states that it performs the appropriate pairwise testing based on the stats input. But I'm trying to find what actual test is used. On the function page it keeps talking about a "hypothesis test", but I can't seem to find which actually test is being used.
In the "More about" section, it talks about t-tests, but I'm not sure if this isn't used as an example or if it means that is simply performs t-tests on the input (which would be performs on the the appropriate type of data, for instance on the ranks when using Kruskal-Wallis?).
Any help is greatly appreciated.
Ive J
Ive J on 5 Dec 2020
Yes it is correct. But the actual test being used depends on the background test; for instance a test like ANOVA would require a t-tes while a non-parametric ones like Kriskal-Wallis would requie a non-parametric one like Wilcoxon. Regardless, the function uses the means or meanranks from stat structure.
[~, ~, stats] = anova1(MPG, Origin, 'off')
stats =
struct with fields:
gnames: {6×1 cell}
n: [64 15 9 3 2 1]
source: 'anova1'
means: [21.1328 31.8000 28.4444 23.6667 22.5000 28]
df: 88
s: 7.0513
[~, ~, stats] = kruskalwallis(MPG, Origin, 'off')
stats =
struct with fields:
gnames: {6×1 cell}
n: [64 15 9 3 2 1]
source: 'kruskalwallis'
meanranks: [38.6719 75.2000 62.1111 49.3333 43.7500 67.5000]
sumt: 1266
You can check this further by
edit multcompare
So, you can realize that the function calculates critical values using tinv (t inverse CDF) for parametric tests (anova1 or anova2), while it uses norminv (normal inverse CDF) for non-parametric (KW or Friedman) ones.

Sign in to comment.

Answers (1)

Scott MacKenzie
Scott MacKenzie on 30 Jul 2021
The pairwise comparisons test used by default with multcompare is Tukey-Kramer. Using the ctype option, you can change the test to Bonferroni, Scheffe, Dunn-Sidak, or Fisher's LSD. Generally, the different tests yield the same or similar outcomes.
The discussion on t-tests in "More About" has nothing to do with how the tests are done in multcompare. This discussion, which I just read, is a nicely written explanation on why a customized test, such as Tukey-Kramer, is needed in the first place. The key point is this: When there are many group means, there are also many pairs to compare. If you applied an ordinary t-test in this situation, the alpha value would apply to each comparison, so the chance of incorrectly finding a significant difference would increase with the number of comparisons. All the pairwise comparisons tests accommodate the latter issue (incorrectly finding a significant difference) by applying a more conservative criterion than would exist if a simple t-test were used for each pairwise comparison.
  1 Comment
Hamed Nassar
Hamed Nassar on 7 Nov 2021 at 19:03
Tukey-Kramer, Bonferroni, Scheffe, Dunn-Sidak, etc. are not statistical tests. They are post-hoc correction schemes to adjust the p-value resulting from the original test, being Anov1, ttest, Freidman, etc.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!