Table of Contents
Repeated measures models, also known as within-subjects designs, are commonly used in research to analyze the effects of a certain variable over time or under different conditions. In these types of designs, the same participants are measured multiple times, making it important to consider the within-subject correlation. One way to analyze the effects in repeated measures models is by testing simple effects. This involves breaking down the overall effect into smaller, specific effects for each level of the independent variable. This can be done using statistical tests, such as ANOVA or t-tests, to compare the means of the dependent variable at each level of the independent variable. By testing simple effects, researchers can gain a deeper understanding of how the independent variable influences the dependent variable within the context of a repeated measures design.
How can I test simple effects in repeated measures models? | Stata FAQ
Testing simple effects in repeated measures models that have both between-subjects and
within-subjects effects can be tricky. We will look at two different estimation
approaches, linear mixed model and anova. The example we will use is a split-plot factorial with
a two-level between variable (a) and a four-level within variable (b). There are four
subjects at each level of a.
We begin by loading the data and producing some descriptive statistics.
use https://stats.idre.ucla.edu/stat/data/spf24, cleartab1 a b s-> tabulation of a a | Freq. Percent Cum. ------------+----------------------------------- 1 | 16 50.00 50.00 2 | 16 50.00 100.00 ------------+----------------------------------- Total | 32 100.00 -> tabulation of b b | Freq. Percent Cum. ------------+----------------------------------- 1 | 8 25.00 25.00 2 | 8 25.00 50.00 3 | 8 25.00 75.00 4 | 8 25.00 100.00 ------------+----------------------------------- Total | 32 100.00 -> tabulation of s s | Freq. Percent Cum. ------------+----------------------------------- 1 | 4 12.50 12.50 2 | 4 12.50 25.00 3 | 4 12.50 37.50 4 | 4 12.50 50.00 5 | 4 12.50 62.50 6 | 4 12.50 75.00 7 | 4 12.50 87.50 8 | 4 12.50 100.00 ------------+----------------------------------- Total | 32 100.00tabstat y, by(a)Summary for variables: y by categories of: a a | mean ---------+---------- 1 | 5.6875 2 | 5.0625 ---------+---------- Total | 5.375 --------------------tabstat y, by(a)/b> Summary for variables: y by categories of: a a | mean ---------+---------- 1 | 5.6875 2 | 5.0625 ---------+---------- Total | 5.375 -------------------- egen ab=group(a b), labelSummary for variables: y by categories of: ab (group(a b)) ab | mean -------+---------- 1 1 | 3.75 1 2 | 4 1 3 | 7 1 4 | 8 2 1 | 1.75 2 2 | 3 2 3 | 5.5 2 4 | 10 -------+---------- Total | 5.375 ------------------tabstat y, by(ab)
Let’s begin with a linear mixed model using the xtmixed command.
We will need to specify the reml option so that the results are consistent
with the anova command that we will run later. Starting with Stata 12
the default estimation method is mle, which is why we need to specify
the reml option.
mixed y a##b || s:, reml stddevPerforming EM optimization: Performing gradient-based optimization: Iteration 0: log restricted-likelihood = -34.824381 Iteration 1: log restricted-likelihood = -34.824379 Computing standard errors: Mixed-effects REML regression Number of obs = 32 Group variable: s Number of groups = 8 Obs per group: min = 4 avg = 4.0 max = 4 Wald chi2(7) = 423.89 Log restricted-likelihood = -34.824379 Prob > chi2 = 0.0000 ------------------------------------------------------------------------------ y | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- 2.a | -2 .6208193 -3.22 0.001 -3.216783 -.7832165 | b | 2 | .25 .5034603 0.50 0.619 -.736764 1.236764 3 | 3.25 .5034603 6.46 0.000 2.263236 4.236764 4 | 4.25 .5034603 8.44 0.000 3.263236 5.236764 | a#b | 2 2 | 1 .7120004 1.40 0.160 -.3954951 2.395495 2 3 | .5 .7120004 0.70 0.483 -.8954951 1.895495 2 4 | 4 .7120004 5.62 0.000 2.604505 5.395495 | _cons | 3.75 .4389855 8.54 0.000 2.889604 4.610396 ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval] -----------------------------+------------------------------------------------ s: Identity | sd(_cons) | .513701 .2233302 .2191052 1.204393 -----------------------------+------------------------------------------------ sd(Residual) | .7120004 .1186667 .5135861 .9870682 ------------------------------------------------------------------------------ LR test vs. linear regression: chibar2(01) = 3.30 Prob >= chibar2 = 0.0346contrast a##b// test main effects and interaction Contrasts of marginal linear predictions Margins : asbalanced ------------------------------------------------ | df chi2 P>chi2 -------------+---------------------------------- y | a | 1 2.00 0.1573 | b | 3 383.67 0.0000 | a#b | 3 38.22 0.0000 ------------------------------------------------
The results above indicate that the a main effect is not significant.
Both the b main effect and the a#b interaction
are significant.
The results of the contrast displayed are displayed as chi-square. We will divide each chi-square
by its degrees of freedom so that the results are scaled as F-ratios (we will not do the
division when df=1).
/* scale as F-ratio*/display 383.67/3127.89display 38.22/312.74
Next, we will run the margins command followed by
marginsplot so that we
can plot the interaction.
margins a#b, vsquishAdjusted predictions Number of obs = 32 Expression : Linear prediction, fixed portion, predict() ------------------------------------------------------------------------------ | Delta-method | Margin Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- a#b | 1 1 | 3.75 .4389855 8.54 0.000 2.889604 4.610396 1 2 | 4 .4389855 9.11 0.000 3.139604 4.860396 1 3 | 7 .4389855 15.95 0.000 6.139604 7.860396 1 4 | 8 .4389855 18.22 0.000 7.139604 8.860396 2 1 | 1.75 .4389855 3.99 0.000 .8896042 2.610396 2 2 | 3 .4389855 6.83 0.000 2.139604 3.860396 2 3 | 5.5 .4389855 12.53 0.000 4.639604 6.360396 2 4 | 10 .4389855 22.78 0.000 9.139604 10.8604 ------------------------------------------------------------------------------ marginsplotmarginsplot, x(b)
At last we are ready to look at the tests of simple effects. We will look at both a for each level
of b anf for b at each level of a. These computations will be done using the
contrast command. Running contrast after mixed
displays the results as chi-square. We will divide each of the chi-square values by its degrees of freedom
to scale the results as a F-ratio, for later comparison to our anova results.
contrast a@b// simple effects for a at each level of b Contrasts of marginal linear predictions Margins : asbalanced ------------------------------------------------ | df chi2 P>chi2 -------------+---------------------------------- y | a@b | 1 | 1 10.38 0.0013 2 | 1 2.59 0.1072 3 | 1 5.84 0.0157 4 | 1 10.38 0.0013 Joint | 4 40.22 0.0000 ------------------------------------------------contrast b@a// simple effects for b at each level of a Contrasts of marginal linear predictions Margins : asbalanced ------------------------------------------------ | df chi2 P>chi2 -------------+---------------------------------- y | b@a | 1 | 3 107.88 0.0000 2 | 3 314.01 0.0000 Joint | 6 421.89 0.0000 ------------------------------------------------ /* scale as F-ratio */display 107.88/335.96display 314.01/3104.67
The raw p-values for a@b indicate that a at b1,
b3 and b4 are significant. For, b@a both simple effects are
significant using the raw p-values. Test of simple effects are a type of post-hoc
procedure and need to be adjusted. We won’t go into the adjustment process on this
page other than to state that there are at least four methods found in the literature
(Dunn’s procedure, Marascuilo & Levin, per family error rate or simultaneous test procedure).
Next we will use the anova command to analyze the repeated measures model.
anova y a / s|a b a#b, repeated(b)Number of obs = 32 R-squared = 0.9613 Root MSE = .712 Adj R-squared = 0.9333 Source | Partial SS df MS F Prob > F -----------+---------------------------------------------------- Model | 226.375 13 17.4134615 34.35 0.0000 | a | 3.125 1 3.125 2.00 0.2070 s|a | 9.375 6 1.5625 -----------+---------------------------------------------------- b | 194.5 3 64.8333333 127.89 0.0000 a#b | 19.375 3 6.45833333 12.74 0.0001 | Residual | 9.125 18 .506944444 -----------+---------------------------------------------------- Total | 235.5 31 7.59677419 Between-subjects error term: s|a Levels: 8 (6 df) Lowest b.s.e. variable: s Covariance pooled over: a (for repeated variable) Repeated variable: b Huynh-Feldt epsilon = 0.9432 Greenhouse-Geisser epsilon = 0.5841 Box's conservative epsilon = 0.3333 ------------ Prob > F ------------ Source | df F Regular H-F G-G Box -----------+---------------------------------------------------- b | 3 127.89 0.0000 0.0000 0.0000 0.0000 a#b | 3 12.74 0.0001 0.0002 0.0019 0.0118 Residual | 18 ----------------------------------------------------------------
The results in the anova table above agree with results from mixed once the chi-squares have
been rescaled as F-ratios. Computing the simple effects after the anova is a more complex process than
we used above. One reason for this is that the two types of simples effects, a@b and b@a,
involve different error terms.
Let’s start with a@b. The sums of square for simple effects for a@b total up to SSa +
SSa#b. The test of a uses s|a as the error term while test of a#b use the residual.
The recommendation for testing simple effects for a@b are to pool s|a and residual.
We can accomplish this by running the anova without the s|a term. After the anova
we will use the contrast command to get the simple effects.
anova y a b a#bNumber of obs = 32 R-squared = 0.9214 Root MSE = .877971 Adj R-squared = 0.8985 Source | Partial SS df MS F Prob > F -----------+---------------------------------------------------- Model | 217 7 31 40.22 0.0000 | a | 3.125 1 3.125 4.05 0.0554 b | 194.5 3 64.8333333 84.11 0.0000 a#b | 19.375 3 6.45833333 8.38 0.0006 | Residual | 18.5 24 .770833333 -----------+---------------------------------------------------- Total | 235.5 31 7.59677419contrast a@bContrasts of marginal linear predictions Margins : asbalanced ------------------------------------------------ | df F P>F -------------+---------------------------------- a@b | 1 | 1 10.38 0.0036 2 | 1 2.59 0.1203 3 | 1 5.84 0.0237 4 | 1 10.38 0.0036 Joint | 4 7.30 0.0005 | Residual | 24 ------------------------------------------------
Once again the F-values are the same for this analysis as for the contrast command used in the
mixed analysis. You will note that the p-values are different. The p-values above
are from an F-distribution with 1 and 24 degrees of freedom. The p-values from the mixed
analysis are distributed as a chi-square with 1 degree of freedom.
Now we need to compute the simple effects for b@a. To do this we will quietly rerun the anova
model followed by the margins command with the within option to see what our cell means are.
quietly anova y a / s|a b a#b margins b, within(a)Predictive margins Number of obs = 32 Expression : Linear prediction, predict() within : a Empty cells : reweight ------------------------------------------------------------------------------ | Delta-method | Margin Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- a#b | 1 1 | 3.75 .3560002 10.53 0.000 3.052253 4.447747 1 2 | 4 .3560002 11.24 0.000 3.302253 4.697747 1 3 | 7 .3560002 19.66 0.000 6.302253 7.697747 1 4 | 8 .3560002 22.47 0.000 7.302253 8.697747 2 1 | 1.75 .3560002 4.92 0.000 1.052253 2.447747 2 2 | 3 .3560002 8.43 0.000 2.302253 3.697747 2 3 | 5.5 .3560002 15.45 0.000 4.802253 6.197747 2 4 | 10 .3560002 28.09 0.000 9.302253 10.69775 ------------------------------------------------------------------------------
We will compute the simple effects of b@a using the contrast command.
contrast b@aContrasts of marginal linear predictions Margins : asbalanced ------------------------------------------------ | df F P>F -------------+---------------------------------- b@a | 1 | 3 35.96 0.0000 2 | 3 104.67 0.0000 Joint | 6 70.32 0.0000 | Residual | 18 ------------------------------------------------
Once again, our results from the anova agree with the results for the mixed.
We see that each of the tests of simple effects are distributed as F with 3 and 18 degrees of freedom.
We also note that the results agree with the mixed results above once they are scaled from
chi-square to F-ratios.
At this point, it would be nice to run pairwise comparisons among each of the levels of b at
each level of a. We can do this using the pwcompare command.
We want the pairwise comparisons adjusted using Tudkey’s HSD. We get this using the mcompare
option. Please note that you cannot get the Tukey adjustment from xtmixed models.
/* pairwise comparisons for a = 1 */pwcompare b#i(1).a, mcompare(tukey) effectsPairwise comparisons of marginal linear predictions Margins : asbalanced --------------------------- | Number of | Comparisons -------------+------------- b#a | 6 --------------------------- --------------------------------------------------------------------------------- | Tukey Tukey | Contrast Std. Err. t P>|t| [95% Conf. Interval] ----------------+---------------------------------------------------------------- b#a | (2 1) vs (1 1) | .25 .5034602 0.50 0.959 -1.172925 1.672925 (3 1) vs (1 1) | 3.25 .5034602 6.46 0.000 1.827075 4.672925 (4 1) vs (1 1) | 4.25 .5034602 8.44 0.000 2.827075 5.672925 (3 1) vs (2 1) | 3 .5034602 5.96 0.000 1.577075 4.422925 (4 1) vs (2 1) | 4 .5034602 7.95 0.000 2.577075 5.422925 (4 1) vs (3 1) | 1 .5034602 1.99 0.230 -.4229247 2.422925 --------------------------------------------------------------------------------- /* pairwise comparisons for a = 2 */pwcompare b#i(2).a, mcompare(tukey) effectsPairwise comparisons of marginal linear predictions Margins : asbalanced --------------------------- | Number of | Comparisons -------------+------------- b#a | 6 --------------------------- --------------------------------------------------------------------------------- | Tukey Tukey | Contrast Std. Err. t P>|t| [95% Conf. Interval] ----------------+---------------------------------------------------------------- b#a | (2 2) vs (1 2) | 1.25 .5034602 2.48 0.097 -.1729247 2.672925 (3 2) vs (1 2) | 3.75 .5034602 7.45 0.000 2.327075 5.172925 (4 2) vs (1 2) | 8.25 .5034602 16.39 0.000 6.827075 9.672925 (3 2) vs (2 2) | 2.5 .5034602 4.97 0.001 1.077075 3.922925 (4 2) vs (2 2) | 7 .5034602 13.90 0.000 5.577075 8.422925 (4 2) vs (3 2) | 4.5 .5034602 8.94 0.000 3.077075 5.922925 ---------------------------------------------------------------------------------
Reference
Kirk, Roger E. (1995) Experimental Design: Procedures for the Behavioral Sciences,
Third Edition. Monterey, California: Brooks/Cole Publishing.
Cite this article
stats writer (2024). How can I test simple effects in repeated measures models?. PSYCHOLOGICAL SCALES. Retrieved from https://scales.arabpsychology.com/stats/how-can-i-test-simple-effects-in-repeated-measures-models/
stats writer. "How can I test simple effects in repeated measures models?." PSYCHOLOGICAL SCALES, 1 Jul. 2024, https://scales.arabpsychology.com/stats/how-can-i-test-simple-effects-in-repeated-measures-models/.
stats writer. "How can I test simple effects in repeated measures models?." PSYCHOLOGICAL SCALES, 2024. https://scales.arabpsychology.com/stats/how-can-i-test-simple-effects-in-repeated-measures-models/.
stats writer (2024) 'How can I test simple effects in repeated measures models?', PSYCHOLOGICAL SCALES. Available at: https://scales.arabpsychology.com/stats/how-can-i-test-simple-effects-in-repeated-measures-models/.
[1] stats writer, "How can I test simple effects in repeated measures models?," PSYCHOLOGICAL SCALES, vol. X, no. Y, ص Z-Z, July, 2024.
stats writer. How can I test simple effects in repeated measures models?. PSYCHOLOGICAL SCALES. 2024;vol(issue):pages.