[HELP] Statistics in MATLAB - Two way Mixed anova or something else ?

11 views (last 30 days)
Hello everyone,
I would like to ask some questions about appropriate statistical test for our study design.
This study aims to evaluate the postural balance among parkinsonian patients.
We have 3 groups, and in each group we have 4 – 4 – 3 subjects, randomly attributed, and 3 different interventions for each group.
We perform a pre-evaluation and a post evaluation after 6 weeks of intervention.
We have severals dependant variables.
As Laerd statistics advise us with the statistical selector we used a Two-way mixed Anova. We have some concern about our statistical power due to the small sample size in each group.
We would like to see group differences, but also if possible, associations & correlation. Is a Two-Way Mixed Anova the right test or Linear Regression is a better one ? Or neither ?
Thank you everyone
  5 Comments
John Doe
John Doe on 15 Aug 2022
You say you're not statistician, but you seems to be like a very good amateur !
I will follow your advice, ask someone who's used to do statistics and consult CrossValidated, and on the other hand, consult the book you quote.
Yes, in fact we only want to see if 1 group (intervention/treatment) is better than the other 2.
Thanks for your answers !

Sign in to comment.

Accepted Answer

William Rose
William Rose on 15 Aug 2022
@John Doe, You're welcome. Thank you for your kind comment.
To determine if treatment 1 is better than treatment 2 or treatment 3, proceed in two steps.
Step 1. Test if treatments 2 and 3 are comparable. If they are comparable (more on that below), then combine groups 2 and 3, and proceed to step 2A. If treatments 2 and 3 are not comparable, proceed to step 2B.
Step 2A: One-sided test performed on the pre-to-post change in score. The null hypothesis is H0: The change in score with Treatment 1 is less than or equal to the change with combined treatment 2 or 3 (which are combined into a single group). You will have a better chance of detecting that Treatment 1 is superior if you use the ones-sided test, since the critical value for rejecting H0 is lower with a 1-sided test. Matlab's t test uses Welch's version of the t test by default, which is good, since it does not depend on equal variances. See here for more on the benefits of Welch's t test.
Step 2B: Do a one-sided test on the change in scores in group 1 versus group 2. The null hypothesis is H0: The change in scores is the same for treatments 1 and 2. Do a one-sided test on group 1 versus group 3, with an analogous null hypothesis.
Now let's return to step 1, deciding if groups 2 and 3 produce comparable results. If we fail to combine groups that are in fact similar (let's call this error A), we will lose power to detect potential superiority of treatment 1, because the numbers in groups 2 and 3 are very low. If we combine groups that are in fact not similar (error B), we will lose power, because the variance of the combined group will be high, and this high variance will make it harder to detect potential superiority of treatment 1. We don't want to make error A or error B. Therefore it is not obvious which null hypotheses is preferable, when testing the equality of groups 2 and 3. We can go with H0a: Groups 2 and 3 are the same. With this null hypothesis, we are unlikely to make error A, because we will only split groups 2 and 3 if there is strong evidence that they are unequal. Or we can go with H0b: Groups 2 and 3 are not the same. With H0b, we are unlikely to make error B, because we will only combine groups if there is strong evidence that they are equal. You will have to decide which null hypothesis you prefer. If you choose H0a, then use a two-sided Welch's t test, discussed above, applied to the change in scores in group 2 versus group 3. If you choose H0b, then use two one-sided tests (TOST), described here (2800 citations!). An advantage of H0a compared to H0b is that with H0a, you do not have to choose a "range of equivalence" or justify that choice, like you have to do with TOST.
The small sample sizes in this study, n=4, 4, 3 for the groups, makes me suspect that no matter how you test, you will be hard-pressed to find significance, at least if you deal in a proper way with the multiple comparisons problem associated with 15-20 dependent variables.
Good luck.

More Answers (1)

William Rose
William Rose on 14 Aug 2022
@John Doe, Please consider my second comment below my answer. I meant to post it as an answer.

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!