Computational Thinking Test for Lower Primary (CTtLP)

Abstract

The Computational Thinking Test for Lower Primary (CTtLP; Zhang & Wong, 2023) is a collection of constructed-response items designed to evaluate computational thinking (CT) abilities in students in lower primary school (Grades 1-3, ages 6-10). The creation of the 27-item CTtLP was guided by an evidence-centered design (ECD), and its psychometric properties were evaluated using classical test theory (CTT) and item response theory (IRT). The items were conceived through three scenarios that evaluated students’ capacity to recognize the output of instruction sequences and to spot errors in a series of instruction sequences. Following an expert review, cognitive interviews, field tests, and a pilot study, 27 of the initial 30 items were kept. The final version of the CTtLP underwent psychometric testing using a sample of students in Grades 1-3 from two public primary schools in northern China. Students were given 60 minutes to finish the test. The results of the reliability, validity, and factor structure were reported.

Keywords

Classical Test Theory; Computational Thinking; Computer Sciences; Early Childhood Education; Error Identification; Evidence Centered Design; Instruction Sequence; Lower Primary Students

Authors

Zhang, Shuhan; Wong, Gary K. W.

---

Purpose

The purpose of this test is to measure computational thinking acquisition in young children.

Validity

Criterion Validity: Correlations of students’ course performance with their CTtLP scores were calculated, and results showed that the two measurements were moderately correlated (r=0.443, p<0.000), indicating a fair criterion validity of CTtLP.
Test Validity: The 3PL model was applied and all items were retained. Three parameters were generated for each item: item discrimination, item difficulty, and guessing rate. The results of the CTtLP showed that the mean difficulty index for the CTtLP items was 0.353 (range=−1.096, 1.892). All items suggested acceptable difficulty, and the test covered a broad range of difficulty levels. Item discrimination yielded a mean of 2.188 (range=1.437, 3.331). In terms of the guessing parameter, the mean of all items was 0.136 (range= 0.001, 0.306). All items yielded low guessing rates below the threshold of 0.35.

Reliability

Internal Consistency: Cronbach’s alpha was 0.873.
Test-Retest Reliability: Using an 8-week interval, intraclass correlation coefficient was 0.757.
Test Reliability: The test information function in the IRT analysis was used to examine the reliability of the test (Aesaert et al., 2014). The information provided by the test was 14.65 when the ability of a student was around 0.9. This indicated that CTtLP provided the most information about the participants with marginally higher-than-average ability levels. In general, CTtLP provided good coverage of a broad range of ability levels.

Factor Analysis

Confirmatory Factor Analysis: To examine the fit of a unidimensional model, a single-factor model was input, and it was found that RMSEA=0.041, CFI=0.911, and TLI=0.900, indicating a proper fit (Hu & Bentler, 1999). Therefore, the unidimensionality of the data set was achieved.

Instrument: Computational Thinking Test for Lower Primary (CTtLP)

Test Type: Original
Format: Responses are provided using a multiple-choice format. Five choices are provided, with one key, three distractors, and an additional option as ‘I don’t know’, which is employed to reduce the likelihood of guessing.
Language Available: English; Chinese
Population Group: Human; Male; Female
Age Group: Childhood (birth-12 yrs); School Age (6-12 yrs)
Population Details:

• Age Range: 6-10 Yrs.

• Location: China

• Respondents: Lower Primary School Students (Grades 1-3)
  Test Methodology: Test Validity; Content Validity; Criterion Validity; Test Reliability; Internal Consistency; Test-Retest Reliability; Factor Analysis; Confirmatory Factor Analysis; Item Response Theory

Keywords

Classical Test Theory; Critical Thinking; Logical Thinking; Primary School Students; Computational Modeling; Computer Science; Executive Functioning Measures; Psychoeducational Assessment; Early Childhood Education

Authors

Author ORCID Identifier:

• Zhang, Shuhan: http://orcid.org/0000-0002-6493-9979

• Wong, Gary K. W.: http://orcid.org/0000-0003-1269-0734

Affiliation:

• Zhang, Shuhan: Faculty of Education, University of Hong Kong

• Wong, Gary K. W.: Faculty of Education, University of Hong Kong

Email addresses:

• Zhang, Shuhan: [email protected]

Correspondence Address:

• Zhang, Shuhan: University of Hong Kong, Faculty of Education, Pokfulam Road, Hong Kong, China, [email protected]

Permissions & Fee and Test Year

Permissions: Contact Corresponding Author
Fee: No
Test Year: 2023

References

Zhang, S., & Wong, G. K. W. (2023). Development and validation of a computational thinking test for lower primary school students. Educational Technology Research and Development, 71(4), 1595–1630. https://doi.org/10.1007/s11423-023-10231-2

Items of the Computational Thinking Test for Lower Primary (CTtLP)

This measure consists of 27 items. No data is Available.