Coaching Fidelity Scale (CFS)

Abstract

The Coaching Fidelity Scale (CFS; Meadan et al., 2023) serves as an observational instrument designed to assess the fidelity with which professionals implement coaching practices. This 11-item checklist is structured around a coaching model employed in previous research (e.g., Meadan et al., 2020), encompassing three distinct phases: (a) pre-observation, which involves discussing the target strategy or skills, identifying the interaction context, and reviewing the target strategy/skill steps; (b) observation, where the uninterrupted interaction between the coach (e.g., service provider, teacher, board-certified behavior analysts (BCBAs), researcher) and the coachee (e.g., caregivers, paraprofessional, registered behavior technicians (RBTs)) is observed; and (c) post-observation, which includes a discussion of the coachee’s reflection, feedback provided by the coach, and joint problem-solving. The data utilized for the analyses were derived from three different intervention studies (Chung et al., 2020, 2021; Meadan et al., 2020). Based on feedback from two of these studies (Chung et al., 2020; Meadan et al., 2020), two additional items (Problem-Solving items 2 and 4) were incorporated into the initial 9-item CFS. An exploratory factor analysis confirmed a unidimensional solution. The study reported results pertaining to reliability (internal consistency and interrater reliability), validity (test sensitivity), and factor structure.

Keywords

Autism; Caregivers; Coaching Practices with Caregivers; Coaching Professionals; Developmental Disabilities; Evidence-Based Practices; Observational Measure; Young Children

Authors

Meadan, Hedda; Lee, James D.; Sands, Michelle M.; Chung, Moon Y.; García-Grau, Pau

Purpose

The primary purpose of the Coaching Fidelity Scale (CFS) is to offer an observational tool specifically designed to evaluate the fidelity of coaching.

Validity

Test Sensitivity: Independent sample comparisons were performed between baseline (pre-intervention observations; n = 50) and post-intervention (n = 81) groups. The results from the Mann–Whitney test indicated a statistically significant difference, with large effect sizes, between pre-intervention (M = 1, SD = 1.20) and post-intervention (M = 5.91, SD = 2.52) scores on the CFS. The post-intervention scores were notably higher (W = 327.00, p < .001, rank-biserial correlation = .84), demonstrating the scale’s sensitivity to change following intervention.

Reliability

Internal Consistency: The internal consistency of the CFS was assessed using Cronbach’s alpha, which was found to be 0.91 (95% CI [0.89, 0.93]). The average inter-item correlations were calculated at 0.47 (95% CI [0.41, 0.53]), indicating a good level of consistency among the items.

Interrater Reliability: The Interclass Correlation Coefficient (ICC) value for the overall CFS score assigned by both observers was above 0.96, demonstrating excellent interrater reliability.

Factor Analysis

Exploratory Factor Analysis: An exploratory factor analysis conducted with 9 items suggested a single factor, with all items loading above 0.76. This factor accounted for 77.84% of the total variance. The overall model exhibited a good fit, indicated by the following statistics: χ²(N = 27) = 9.423, p = .99, CFI = 0.981, GFI = 0.975, TLI = 0.975, RMSEA = 0.065, RMSR = 0.0860, WRMR = 0.0449. Furthermore, all item loadings exceeded 0.76 (JP1), and communalities were above 0.609. No noteworthy cross-loadings were identified. These findings strongly support a unidimensional solution for the analysis, which aligns with the theoretical objective of the CFS: that all items collectively measure a single global score of coaching fidelity.

Instrument: Coaching Fidelity Scale (CFS)

Test Type: Original
Format: The items are presented in a checklist format, consisting of yes/no questions based on observation. An optional step for video feedback is included, allowing the coach and coachee to jointly review and discuss a video recording from a previous session.
Language Available: English
Population Group: Human (Male; Female)
Age Group: Adulthood (18 years and older)
Population Details: The respondents for this measure are professionals and caregivers of young children (under 5 years old) who have autism or other developmental disabilities.
Test Methodology: The methodology employed for evaluating the CFS included Test Reliability, Internal Consistency, Interrater Reliability, Factor Analysis, Exploratory Factor Analysis, and Test Sensitivity.

Keywords

Autism; Caregivers; Coaching Practices with Caregivers; Coaching Professionals; Developmental Disabilities; Evidence-Based Practices; Observational Measure; Young Children

Authors

Author ORCID Identifier: No data is Available

Affiliation Email addresses:
Hedda Meadan: [email protected]
James D. Lee: No data is Available
Michelle M. Sands: No data is Available
Moon Y. Chung: No data is Available
Pau García-Grau: No data is Available

Correspondence Address:
Meadan, Hedda: University of Illinois Urbana-Champaign, Department of Special Education, 1310 South Sixth St., Champaign, Illinois, United States, 61820, [email protected]

Permissions & Fee and Test Year

Permissions: Contact Corresponding Author for permission.
Commercial Use: No
Fee: No
Test Year: 2023

References

Meadan, H., Lee, J. D., Sands, M. M., Chung, M. Y., & García-Grau, P. (2023). The Coaching Fidelity Scale (CFS): Development and evaluation of an observational measure of coaching fidelity. Infants & Young Children, 36(1), 37–52. https://doi.org/10.1097/IYC.0000000000000231

Items of the Coaching Fidelity Scale (CFS)

The Coaching Fidelity Scale (CFS) consists of 11 items, with an additional optional video feedback item. The specific items are not provided, but they are designed to evaluate the extent to which professionals implement a set of coaching practices with fidelity, following a model that includes pre-observation, observation, and post-observation phases. Two specific problem-solving items (Problem-Solving items 2 and 4) were added to the initial 9-item CFS based on feedback from prior studies.