3-way Repeated Measures ANOVA pairwise comparisons using multcompare

I'm expanding on a helpful question/answer I found here: http://www.mathworks.com/matlabcentral/answers/124353-does-fitrm-ranova-support-within-subject-models-without-between-subject-factors

I have run a 3-way repeated measures ANOVA with 3 within subject factors. I want to make the pairwise comparisons for the 2- and 3-way interactions; I can run these for the 2-way interactions using the method RepeatedMeasuresModel/multcompare but I cannot figure out how to do it for the 3-way interaction. Here is my example:

% generate random data for the example
alpha_power = randn(24,8);

% Create a table storing the respones
varNames = {'Y1','Y2','Y3','Y4','Y5','Y6','Y7','Y8'};
t = array2table(alpha_power,'VariableNames',varNames);

% Create a table reflecting the within subject factors 'TestCond', 'Attention', and 'TMS' and their levels
factorNames = {'TestCond','Attention','TMS'};
within = table({'M';'M';'M';'M';'N';'N';'N';'N'},{'A';'A';'V';'V';'A';'A';'V';'V'},{'T';'S';'T';'S';'T';'S';'T';'S'},'VariableNames',factorNames);

% fit the repeated measures model
rm = fitrm(t,'Y1-Y8~1','WithinDesign',within);

% run my repeated measures anova here
[ranovatbl] = ranova(rm, 'WithinModel','TestCond*Attention*TMS');

% make pairwise comparisons for the two-way interactions
%
% see: help RepeatedMeasuresModel/multcompare
multcompare(rm,'TestCond','By','Attention')
multcompare(rm,'TestCond','By','TMS')
multcompare(rm,'Attention','By','TMS')

% but how can I make pairwise comparisons for the 3-way interaction?
%
% this does not work (it ignores the 'Attention' factor)
multcompare(rm,'TestCond','By','Attention','By','TMS')

I found this old discussion, but it was not helpful (it refers to the standalone multcompare function): http://www.mathworks.com/matlabcentral/answers/2141-how-to-obtain-p-values-for-all-pair-wise-comparisons-from-the-multicompare-function

Is it possible to test the pairwise comparisons in the 3-way interaction?

[Richard Barrett-Jolley]

I hope 6 years on you have resolution to your issue!

...meanwhile, just wanted to flag that your QUESTION solved my issue...

so thanks for sharing your code!

Answer 1

[Matt Mollison]

I received help from the Mathworks support and thought I would post their answer here (thanks to Eric Diaz and colleagues).

% Suppose we want to compare levels of Attention for each combination
% of levels of TestCond and TMS.

% 1. Convert factors to categorical.
within2 = within;
within2.Attention = categorical(within2.Attention);
within2.TestCond = categorical(within2.TestCond);
within2.TMS = categorical(within2.TMS);

% 2. Create an interaction factor capturing each combination of levels
% of TestCond and TMS.
within2.TestCond_TMS = within2.TestCond .* within2.TMS;

% 3. Call fitrm with the modified within design.
rm2 = fitrm(t,'Y1-Y8~1','WithinDesign',within2);
ranovatbl2 = ranova(rm2, 'WithinModel','TestCond*Attention*TMS')

% 4. Use interaction factor TestCond_TMS as the 'By' variable in multcompare.
multcompare(rm2,'Attention','By','TestCond_TMS')

It is also possible to set up a 3-way interaction in a similar way to step 2, run fitrm, and then run multcompare(rm2,'Attention_TestCond_TMS') to get all of the pairwise comparisons (corrected for multiple comparisons).

[Abbie]

Changing the data type to categorical changes the DF in the resulting anova table for me. My factors are actually numerical, so it's not obvious to me which data type is appropriate. Do you have any insight into how or why ranova treats categorical factors differently than non-categorical factors?

[Steven Jerjian]

I've tried this with my data but keep running in to errors on step 4, when running the multcompare. My within2 variable seems to have the correct interaction factors, but I get either "The grouping value name [...] is not recognized as a factor name", or even when I try to run this exact example, I get "Unable to merge the 'Attention' variables in A and B" due to some error with the use of categorical variables. Is there a change in Matlab that could be causing this (I am using 2019b)??

[Óscar Miranda Domínguez]

Thanks foir sharing! That was a clever trick! I made a workaround to avoid casting variables as categoricals:

% Suppose we want to compare levels of Attention for each combination
% of levels of TestCond and TMS.

% 1. Make temp variable to combine columns.
within2 = within;
temp=[within2{:,1} within2{:,2}];% unfold data from  table to cell array
temp=[cat(1,temp{:,1}) repmat(' ',size(within2,1),1) cat(1,temp{:,2})];% concaenate text and add space (the repmat thing...)
temp=cellstr(temp);% I had to convert back to cell since that is the format of the other columns

% 2. Create an interaction factor capturing each combination of levels
% of TestCond and TMS.
to_eval=['within2.' within2.Properties.VariableNames{1} '_' within2.Properties.VariableNames{2} ' = temp;'];% avoid harcoding and generalize for any column names
eval(to_eval);% run the command

% 3. Call fitrm with the modified within design.
rm2 = fitrm(t,'Y1-Y8~1','WithinDesign',within2);
ranovatbl2 = ranova(rm2, 'WithinModel','TestCond*Attention*TMS')

% 4. Use interaction factor TestCond_TMS as the 'By' variable in multcompare.
multcompare(rm2,'Attention','By','TestCond_TMS')

[chao zheng]

It doesn't work.

TestCond_TMS Attention_1 Attention_2 Difference StdErr pValue Lower Upper

____________ ___________ ___________ __________ ______ ______ _____ _____

{'M A'} {'A'} {'V'} NaN NaN 1 NaN NaN

{'M A'} {'V'} {'A'} NaN NaN 1 NaN NaN

{'M V'} {'A'} {'V'} NaN NaN 1 NaN NaN

{'M V'} {'V'} {'A'} NaN NaN 1 NaN NaN

{'N A'} {'A'} {'V'} NaN NaN 1 NaN NaN

{'N A'} {'V'} {'A'} NaN NaN 1 NaN NaN

{'N V'} {'A'} {'V'} NaN NaN 1 NaN NaN

{'N V'} {'V'} {'A'} NaN NaN 1 NaN NaN