Calculating Test Reliability (Composite Reliability) – Chinese Version

Description

In fields such as psychology and education, reliability is crucial when evaluating the quality of a test. It serves as an indicator of the stability and consistency of the test results, and any test development or revision must report its reliability. Test users commonly use the α coefficient to estimate test reliability; however, this estimation is conditional upon the test being fundamentally τ-equivalent and the errors being uncorrelated. If these conditions are not met, the α coefficient may overestimate or underestimate the test reliability. In general research, error correlation is typically not permitted unless theoretically supported. The condition for the test being fundamentally τ-equivalent is challenging to meet, with few tests achieving such status, thus making the α coefficient a poor indicator of test reliability. Traditional textbooks refer to the α coefficient as internal consistency reliability or homogeneity coefficient, which can be misleading. Despite many critiques in the literature regarding the α coefficient, it is still widely used in practice. Factor analysis can provide a better estimate of test reliability through composite reliability. Researchers Wen Zhonglin and Ye Baojuan found that composite reliability is indeed internal consistency reliability, as it measures the correlation among items. They suggest continuing to use the more popular term “composite reliability” to avoid the confusion surrounding “internal consistency reliability.” There is limited research using composite reliability to estimate test reliability in both domestic and international studies, primarily due to the calculation difficulties for general test users.

Authors and Contact Email

Wen Zhonglin, Ye Baojuan. Contact information not available.

Purpose

To discuss the importance of reliability in test evaluation and propose a better method for estimating reliability through composite reliability.

Test Year

Information not available.

Administration Method and Scoring

To calculate composite reliability using SPSS software for unidimensional tests, exploratory factor analysis should first be performed with the required items, fixing the number of factors extracted to one. The output file will provide the λi values, which can then be used to calculate composite reliability through a simple formula. Before calculating composite reliability, a suitability check for exploratory factor analysis is needed, which includes ensuring a KMO value above 0.7 and a Bartlett’s test result of P Reliability and Validity
Composite reliability offers a more accurate estimation of test reliability than the α coefficient under conditions where the assumptions for the latter are not satisfied.

Factors and Subscales

For tests with multiple dimensions, to calculate the composite reliability of a single dimension, only the relevant items for that dimension should be selected during exploratory factor analysis, followed by the same calculation steps as for unidimensional tests. It is important to note that the method described is limited to calculating composite reliability for unidimensional tests or for a single dimension within multidimensional tests and is not suitable for calculating composite reliability for the entire multidimensional test.

Keywords

Reliability, Test Evaluation, α Coefficient, Composite Reliability, Exploratory Factor Analysis

Items in Chinese

Information not available.

Items in English

Information not available.

References

温忠麟,方杰,沈嘉琦,谭倚天,李定欣,马益铭.新世纪20年国内心理统计方法研究回顾[J].心理科学进展,2021,29(08):1331-1344.杨强,叶宝娟,温忠麟.(2014b).用SPSS软件计算单维测验的合成信度.中国临床心理学杂志,22(3),496–498