A one-way ANOVA is used to determine whether or not there is a statistically significant difference between the means of three or more independent groups.

If the overall p-value from the ANOVA table is less than some significance level, then we have sufficient evidence to say that at least one of the means of the groups is different from the others.

However, this doesn’t tell us which groups are different from each other. It simply tells us that not all of the group means are equal.

In order to find out exactly which groups are different from each other, we must conduct a post hoc test.

One of the most commonly used post hoc tests is Scheffe’s test, which allows us to make pairwise comparisons between the means of each group while controlling for the family-wise error rate.

The following example shows how to perform Scheffe’s test in R.

Example: Scheffe’s Test in SAS

Suppose a researcher recruits 30 students to participate in a study. The students are to use one of three studying methods to prepare for an exam.

We can use the following code to create this dataset in SAS:

/*create dataset*/
data my_data;
    input Method $ Score;
    datalines;
A 76
A 77
A 77
A 81
A 82
A 82
A 83
A 84
A 85
A 89
B 81
B 82
B 83
B 83
B 83
B 84
B 87
B 90
B 92
B 93
C 77
C 78
C 79
C 88
C 89
C 90
C 91
C 95
C 98
C 98
;
run;

Next, we’ll use proc ANOVA to perform the one-way ANOVA:

/*perform one-way ANOVA with Scheffe's post-hoc test*/
proc ANOVA data=my_data;class Method;model Score = Method;means Method /scheffe cldiff;
run;

Note: We used the means statement along with the scheffe and cldiff options to specify that Scheffe’s post-hoc test should be performed (with confidence intervals) if the overall p-value of the one-way ANOVA is statistically significant.

First, we’ll analyze the ANOVA table in the output:

From this table we can see:

• The overall F Value: 3.49
• The corresponding p-value: 0.0448

• H₀: All group means are equal.
• H_A: At least one group mean is different from the rest.

Since the p-value from the ANOVA table (0.0448) is less than α = .05, we reject the null hypothesis.

This tells us that the mean exam score is not equal between the three studying methods.

To determine exactly which group means are different, we must refer to the final table in the output that shows the results of Scheffe’s post-hoc tests:

To tell which group means are different, we must look at which pairwise comparisons have stars (***) next to them.

From the table we can see there is a statistically significant difference in mean exam scores between group A and group C.

There are no statistically significant differences between any other group means.

Specifically, we can see that the mean difference in exam scores between group C and group A is 6.7.

The 95% confidence interval for the difference in means between these groups is [0.064, 13.336].

The following tutorials provide additional information about ANOVA models:

A Guide to Using Post Hoc Tests with ANOVA