Classroom Assessment Fairness Inventory (CAFI)

Abstract

The Classroom Assessment Fairness Inventory (CAFI), developed by Rasooli et al. (2023), is designed to interpret students’ perceptions of fairness in classroom assessment. This inventory consists of five distinct scenarios, each addressing a specific domain of classroom assessment crucial to students’ perceptions of fairness, including groupwork, exams, cheating, grading, and feedback. Each scenario is accompanied by a set of actions, totaling 40 actions across all scenarios. These actions are designed to represent an underlying principle of fairness (e.g., equality, equity, transparency) derived from social psychology theory as it pertains to classroom assessment. The CAFI was administered to a sample of first-year undergraduate students in Canada, and comprehensive results regarding its factor structure, reliability, and validity have been reported.

Keywords

Classroom Assessment Fairness; Student Perceptions; Unfairness in Groupwork; Fairness in Cheating; Fairness in Grading; Unfairness in Feedback; Fairness in Feedback; Educational Measures; Feedback; Grading (Educational); Student Attitudes; Teacher Student Interaction; Fairness; Cheating.

Authors

Amirhossein Rasooli, Christopher DeLuca, Liying Cheng, Amin Mousavi

---

Purpose

The primary purpose of the CAFI is to assess students’ perceptions of fairness in assessment practices within a classroom setting.

Validity

The validity of the CAFI was investigated through multivariate regression analysis. The results indicated that students’ personal belief in a just world positively predicted perceived fairness in grading (β = 0.262, p < 0.05) and fairness in feedback (β = 0.294, p < 0.05). Conversely, it negatively predicted perceived unfairness in feedback (β = −0.19, p < 0.05). These findings provide partial empirical evidence supporting the validity of the inventory, suggesting that students who hold a personal belief in a just world tend to perceive greater fairness in grading and feedback processes.

Reliability

Interrater reliability: The overall inter-rater agreement demonstrated a moderate average value of 0.56 (p < 0.001).
Internal consistency: Cronbach’s alpha values for the constructs ranged from 0.79 to 0.87, indicating good internal consistency.

Factor Analysis

Both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) were conducted. The five-factor model showed a better fit with the data, consistent with the EFA results. The overall goodness-of-fit indices for the five-factor model were RMSEA = 0.06 (90% CI [0.05–0.07]), CFI = 0.91, and TLI = 0.89, indicating a satisfactory fit. The six-factor model also showed acceptable fit (RMSEA = 0.07, 90% CI [00.06–0.08], CFI = 0.90, TLI = 0.87); however, the additional factor in this model only retained two action items (S2–6 and S2–7) from Scenario 2, which were deemed conceptually inadequate to represent the exam scenario, leading to its disregard in favor of the five-factor model.

Instrument

Test Type: Original
Format: Students are instructed to read each scenario and then evaluate the extent to which they perceive each associated action as fair, using a 6-point Likert scale (1 = Highly unfair; 6 = Highly fair). An additional option, “7 = ‘I am not sure’,” is also provided.
Language Available: English
Population Group: Human (Male, Female, Does Not Identify with Gender Binary)
Age Group: Childhood (birth-12 yrs), School Age (6-12 yrs), Adolescence (13-17 yrs), Adulthood (18 yrs & older), Young Adulthood (18-29 yrs).
Population Details: The sample consisted of university students aged 15-21 years and older, located in Canada.
Test Methodology: Test Validity; Test Reliability; Internal Consistency; Interrater Reliability; Factor Analysis; Confirmatory Factor Analysis; Exploratory Factor Analysis.

Keywords

Classroom Assessment Fairness; Student Perceptions; Unfairness in Groupwork; Fairness in Cheating; Fairness in Grading; Unfairness in Feedback; Fairness in Feedback.

Authors

Amirhossein Rasooli

• Author ORCID Identifier: No data is Available

• Affiliation: University of Alberta Killam Postdoctoral Fellow

• Email Address: [email protected]

• Correspondence Address: University of Alberta, Faculty of Education, 11210 87 Ave NW, Edmonton, Alberta, Canada, T6G 2G5, [email protected]

Christopher DeLuca

• Author ORCID Identifier: No data is Available

• Affiliation: Queen’s University Faculty of Education

• Email Address: No data is Available

Liying Cheng

• Author ORCID Identifier: http://orcid.org/0000-0002-4458-5085

• Affiliation: Queen’s University Faculty of Education

• Email Address: No data is Available

Amin Mousavi

• Author ORCID Identifier: http://orcid.org/0000-0002-6920-2319

• Affiliation: University of Saskatchewan Faculty of Education

• Email Address: No data is Available

Permissions & Fee and Test Year

Permissions: Contact Publisher
Commercial: No
Fee: No
Test Year: 2023

References

Rasooli, A., DeLuca, C., Cheng, L., & Mousavi, A. (2023). Classroom assessment fairness inventory: A new instrument to support perceived fairness in classroom assessment. Assessment in Education: Principles, Policy & Practice, 30(5-6), 372–395. https://doi.org/10.1080/0969594X.2023.2255936

Items of the Classroom Assessment Fairness Inventory (CAFI)

The CAFI is a 21-item measure. The specific items are designed within five scenarios, each addressing a different classroom assessment domain. The factors and subscales identified from the measure are:

• Unfairness in Groupwork

• Fairness in Cheating

• Fairness in Grading

• Unfairness in Feedback

• Fairness in Feedback

Specific item details are available in the source reference: Rasooli, A., DeLuca, C., Cheng, L., & Mousavi, A. (2023). Classroom assessment fairness inventory: A new instrument to support perceived fairness in classroom assessment. Assessment in Education: Principles, Policy & Practice, 30(5-6), 372–395. Specific items can be found in Table 3, Page 379; Table 4, Page 379; Table 5, Page 380; Table 6, Page 380; Table 7, Page 381.

Classroom Assessment Fairness Inventory (CAFI)

Scenario 1: Groupwork

Mr. Chu implemented group work based on initial assessments, forming groups with students of varying ability levels: struggling, average, and high-performing. He expected high-performing students to contribute to the learning of other group members. Each group worked on projects and prepared a final presentation. Mr. Chu then held a class discussion on workload distribution and allowed students to discuss group dynamics. In response to students’ questions about assessment, he explained that grades would be based on project participation and a rubric, not solely on individual performance. He encouraged group members to collaborate, stating that all members would receive the same grades as a reflection of group performance and cooperation. Several students were not satisfied with their grades and appealed, but Mr. Chu did not accept their complaints.

To what extent do you consider each of these actions to be fair based on the above scenario?

Scenario 2: Exam

Mr. Ahmed announced that the class would have an exam the day before winter break (in 5 days). Students preferred moving the exam date because they had many assignments for other subjects that were also due on the same date. Even so, Mr. Ahmed was firm on his decision as moving the exam date back would create more intensive workload later in the year. Mr. Ahmed did not explicitly state what would be on the exam. However, he did include a mix of easy and difficult questions to give all students an opportunity to show their learning. He also provided accommodations (e.g. more time) to students with disabilities and English language learners. In general, Mr. Ahmed is a lenient teacher in grading compared with other teachers in the school who teach the same subject. On the exam, all students complained about two questions that were not covered during the course. Mr. Ahmed harshly responded that students should be able to answer the two questions from what had been taught. For a few students who missed the exam date, Mr. Ahmed decided to give another exam opportunity after winter break.

To what extent do you consider each of these actions to be fair based on the above scenario?

Scenario 3: Cheating

Ms. Johnston is very strict when she catches a student cheating. However, she did not tell students her policy on cheating at the beginning of the year. One student was caught cheating on an exam and Ms. Johnston decided to give the student a grade of zero. Ms. Johnston did not give the student an opportunity to explain the reasons for cheating before making her decisions. She explained to the class that cheating is unfair to other students and asked the student to leave the classroom. The exam constituted 20% of students’ final grade. After the exam, she met with the student and explained that cheating is ethically wrong, is unfair in relation to the classmates, and she would punish anyone who cheats. The student provided a reason for their behavior and apologized.

To what extent do you consider each of these actions to be fair based on the above scenario?

Scenario 4: Grading

Ms. Mendes had students from diverse backgrounds in her classroom. She treated all her students respectfully during classroom assessment, teaching, and interactions. Ms. Mendes informed students that she would give grades based on student achievement. 70% of students’ grades were from multiple tests during the course plus 30% for individual essays. Ms. Mendes communicated test results in one week after handing the test in. Due to busy schedule, she would sometimes allow students to appeal their grades if there was enough time in class. She would fully explain her grading for students who spoke up looking for their grade adjustments. At the end of the course, Ms. Mendes adjusted the grades of failing students with at-risk backgrounds to support their success. She also increased marks for a few students to ensure admission into their desired universities. However, she lowered the grades of a few disruptive students who interrupted the classroom learning.

To what extent do you consider each of these actions to be fair based on the above scenario?

Scenario 5: Feedback

Mr. Dembe asked students to write an essay about their science lab project. He has shared with students a clear rubric that he will use to assess students’ essays. Four days after the essay submission deadline, Mr. Dembe got back to students with his feedback and expressed that students can contact him for further discussion of his feedback. Students noticed that Mr. Dembe gave more feedback to students with good quality essays as well as his favorite students than students who had handed in essays that were of low quality. Mr. Dembe harshly explained that he had given variable feedback on the essays based on the amount of effort he deemed each student had put in completing the essays.

To what extent do you consider each of these actions to be fair based on the above scenario?

Note: Students are asked to read each scenario and identify the extent to which they consider each action to be fair (1 = Highly unfair; 6 = Highly fair). An additional option (7 = ‘I am not sure’) is also provided.