An (ICC) is used to determine if items (or subjects) can be rated reliably by different raters.

The value of an ICC can range from 0 to 1, with 0 indicating no among raters and 1 indicating perfect reliability.

This tutorial provides a step-by-step example of how to calculate ICC in Excel.

Step 1: Create the Data

Suppose four different judges were asked to rate the quality of 10 different college entrance exams. The results are shown below:

Step 2: Fit an ANOVA

In order to calculate the ICC for these ratings, we first need to fit an Anova: Two-Factor Without Replication.

To do so, highlight cells A1:E11 as follows:

To do so, click the Data tab along the top ribbon and then click the Data Analysis option under the Analysis group:

If you don’t see this option available, you need to first .

In the dropdown menu that appears, click Anova: Two-Factor Without Replication and then click OK. In the new window that appears, fill in the following information and then click OK:

The following results will appear:

Step 3: Calculate the Intraclass Correlation Coefficient

We can use the following formula to calculate the ICC among the raters:

The intraclass correlation coefficient (ICC) turns out to be 0.782.

Here is how to interpret the value of an intraclass correlation coefficient, according to :

• Less than 0.50: Poor reliability
• Between 0.5 and 0.75: Moderate reliability
• Between 0.75 and 0.9: Good reliability
• Greater than 0.9: Excellent reliability

Thus, we would conclude that an ICC of 0.782 indicates that the exams can be rated with “good” reliability by different raters.

A Note on Calculating ICC

There are several different versions of an ICC that can be calculated, depending on the following three factors:

• Model: One-Way Random Effects, Two-Way Random Effects, or Two-Way Mixed Effects
• Type of Relationship: Consistency or Absolute Agreement
• Unit: Single rater or the mean of raters

In the previous example, the ICC that we calculated used the following assumptions:

• Model: Two-Way Random Effects
• Type of Relationship: Absolute Agreement
• Unit: Single rater

For a detailed explanation of these assumptions, please refer to .