Cochran’s Q test is a statistical test that is used to determine whether the proportion of “successes” is equal across three or more groups in which the same individuals appear in each group.

For example, we may use Cochran’s Q test to determine if the proportion of students who pass a test is equal when using three different studying techniques.

Steps to Perform Cochran’s Q Test

Cochran’s Q test uses the following null and alternative hypotheses:

Null Hypothesis (H₀): The proportion of “successes” is the same in all groups

Alternatve Hypothesis(H_A): The proportion of “successes” is different in at least one of the groups

The test statistic is calculated as:

where:

• k: The number of treaments (or “groups”)
• X.j: The column total for the j^th treatment
• b: The number of blocks
• Xi.: The row total for the i^th block
• N: The grand total

The test statistic T follows a Chi-Square distribution with k-1 degrees of freedom.

If the associated with the test statistic is less than a certain significance level (like α = .05), we can reject the null hypothesis and conclude that we have sufficient evidence to say the proportion of “successes” is different in at least one of the groups.

Example: Cochran’s Q Test

Suppose a researcher wants to know if three different studying techniques lead to different proportions of pass rates among students.

To test this, she recruits 20 students to each take an exam of equal difficulty using three different studying techniques. The results are shown below:

Here’s the code we can use to create this dataset and perform Cochran’s Q test in the statistical programming language R:

#load DescTools packagelibrary(DescTools)

#create dataset
df <- data.frame(student=rep(1:20, each=3),
                 technique=rep(c('A', 'B', 'C'), times=20),
                 outcome=c(1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1,
                           1, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1,
                           1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1,
                           1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1))

#perform Cochran's Q test
CochranQTest(outcome ~ technique| student, data=df)

	Cochran's Q test

data:  outcome and technique and student
Q = 0.33333, df = 2, p-value = 0.8465

From the output of the test we can observe the following:

• The test statistic is 0.333
• The corresponding p-value is 0.8465

Since this p-value is not less than .05, we fail to reject the null hypothesis.

This means we do not have sufficient evidence to say that the studying technique used by students leads to different proportions of passing rates.