Table of Contents
Intraclass Correlation Coefficient (ICC) is a measure of reliability between two or more observations of a single group. It can be calculated in Excel using the Analysis Toolpak add-on. First, enter the data into two columns, one for each observation. Then, select ‘Data Analysis’ from the Data tab and choose ‘Correlation.’ Select the two columns of data as the input range, and then select ‘Intraclass’ as the correlation coefficient type. The output will provide the ICC and associated values.
An (ICC) is used to determine if items (or subjects) can be rated reliably by different raters.
The value of an ICC can range from 0 to 1, with 0 indicating no among raters and 1 indicating perfect reliability.
This tutorial provides a step-by-step example of how to calculate ICC in Excel.
Step 1: Create the Data
Suppose four different judges were asked to rate the quality of 10 different college entrance exams. The results are shown below:
Step 2: Fit an ANOVA
In order to calculate the ICC for these ratings, we first need to fit an Anova: Two-Factor Without Replication.
To do so, highlight cells A1:E11 as follows:
To do so, click the Data tab along the top ribbon and then click the Data Analysis option under the Analysis group:
If you don’t see this option available, you need to first .
In the dropdown menu that appears, click Anova: Two-Factor Without Replication and then click OK. In the new window that appears, fill in the following information and then click OK:
The following results will appear:
Step 3: Calculate the Intraclass Correlation Coefficient
We can use the following formula to calculate the ICC among the raters:
The intraclass correlation coefficient (ICC) turns out to be 0.782.
Here is how to interpret the value of an intraclass correlation coefficient, according to :
- Less than 0.50: Poor reliability
- Between 0.5 and 0.75: Moderate reliability
- Between 0.75 and 0.9: Good reliability
- Greater than 0.9: Excellent reliability
Thus, we would conclude that an ICC of 0.782 indicates that the exams can be rated with “good” reliability by different raters.
A Note on Calculating ICC
There are several different versions of an ICC that can be calculated, depending on the following three factors:
- Model: One-Way Random Effects, Two-Way Random Effects, or Two-Way Mixed Effects
- Type of Relationship: Consistency or Absolute Agreement
- Unit: Single rater or the mean of raters
In the previous example, the ICC that we calculated used the following assumptions:
- Model: Two-Way Random Effects
- Type of Relationship: Absolute Agreement
- Unit: Single rater
For a detailed explanation of these assumptions, please refer to .