This FAQ presents an ado program (smecriticalvalue) that can be used to compute critical values for tests
of simple interactions and simple main effects. You can find the smecriticalvalue program by
typing search smecriticalvalue in the Stata command window (see
How can I use the search command to search for programs and get additional
help? for more information about using search).

There are at least four different approaches in the literature for computing critical values for
test of simple interactions and simple main effects. There is a method attributable to Dunn and
one by Marascuilo and Levin. In addition there is the per family error rate and a very conservative
method called the simultaneous test procedure that is closely related to the Scheff´e method
for multiple comparisons. There does not seem to be a clear favorite among these approaches in the literature.

Let’s say that you have run a 3×4 factorial model with a significant interaction. The original 3×4 model
has a total of 11 degrees of freedom for for the anova and 48 for the error term. You follow up the anova by
doing four tests of simple main effects. Each of these tests has 2 degrees of freedom in the numerator and
48 in the denominator. Here is how you can compute the critical values using smecriticalvalue.

smecriticalvalue, number(4) df1(2) df2(48) dfmodel(11)

  number of tests: 4
     numerator df: 2
   denominator df: 48
original model df: 11

Critical value of F for alpha = .05 using ...
------------------------------------------------
Dunn's procedure              = 4.7033201
Marascuilo & Levin            = 5.1923643
per family error rate         = 4.8075721
simultaneous test procedure   = 8.4455227

If you are using the per family error rate then the critical value would be about 4.81.

If you had done the tests of simple main effects the other way around, that is, three tests with 3 degrees
of freedom in the numerator and 48 in the denominator, the critical values would look like this:

smecriticalvalue, number(3) df1(3) df2(48) dfmodel(11)

  number of tests: 3
     numerator df: 3
   denominator df: 48
original model df: 11

Critical value of F for alpha = .05 using ...
------------------------------------------------
Dunn's procedure              = 3.9394001
Marascuilo & Levin            = 4.3040955
per family error rate         = 3.7605452
simultaneous test procedure   = 5.6303484

This time the critical value for the per family error rate is 3.76.

Reference

Kirk, R.E. Experimental design: Procedures for the behavioral sciences, 3rd ed. (1995) Pacific Grove, CA: Brooks/Cole