Assessment Of Computational Thinking Research Articles

Systematic Review of Instruments to Assess Computational Thinking in Early Years of Schooling

Computational thinking (CT) is considered a key competence in today’s digital era. It is an emerging construct that relates to critical thinking and creativity. Research on its assessment is in the process of consolidation. This systematic review aims to analyze studies that have used CT assessment instruments for children and adolescents aged 4 to 16 years in order to identify which variables, they assess and their psychometric properties. The search and analysis were carried out following the PRISMA statement protocol, analyzing 50 articles published between 2006 and March 2023. An increase in the publication of CT measurement instruments is observed, with 54% of them supported by evidence of validity and 88% by reliability, highlighting construct validity, followed by content and criteria validity. China leads in the number of publications, while Asia and Europe concentrate most of the research. There is a noticeable contribution from South America, evidencing the lack of participation from Central and South American countries in this field of study.

Teachers’ understanding of assessing computational thinking

ABSTRACT Background and context Assessment is one of the challenges related to the introduction of Computational thinking (CT) in compulsory education. Objective This study aims to explore teachers’ understanding of CT assessment and the associated possibilities and challenges when CT is integrated into school subjects like mathematics and science. Method We conducted interviews with teachers from Norway and Finland, where CT is included in the national core curricula. The interviews were analysed thematically by adapting a framework for CT integration into subjects. Findings Teachers found CT assessment to be challenging, due to its broader scope beyond coding. Formative assessment was found to be suitable for assessing CT, whereas national exams could become prescriptive amid assessment uncertainties. Implications Definitions of CT and vague curricular goals are not enough for teachers to assess CT. Professional development or clearer assessment guidelines could ameliorate the difficulties with assessing CT..

Development of a Computational Thinking (CT) Module to Help Pre-Service Teachers Learn How to Design CT Instruments

The lack of a Computational Thinking (CT) module, which teaches how to design a CT assessment instrument causes pre-service teachers to be unable to create one. This can cause students at school to not be able to measure their achievements related to CT. This study aimed to develop a CT module to support the ability of pre-service teachers to design CT assessment instruments. This research was a development study that refers to the 4-D model, but the dissemination stage was not carried out. The research subjects were 94 pre-service teachers, 70 female students and 24 male students majoring in Biology Education in semester 4 of the academic year 2021/2022. This research instrument used a validation sheet by material experts and linguists and the test item analysis sheet with 13 criteria made by the directorate of high school development in 2010. The data analysis technique used in this study is a quantitative descriptive analysis technique. The results showed that the conclusions from the validation of the modules developed from the material and presentation aspects were an average of 88.94. The language and readability aspects were an average of 80.56, and both were in the very good category. In contrast, the results of the item analysis on the material, construction and language/culture aspects had a score of 77.45%, 50.29% and 84.71%. The average percentage of ability in designing assessment instruments was 70.82%. This study concluded that the module that has been developed is suitable for use and can support the ability of pre-service teachers to design CT assessment instruments. Keywords: computational thinking, module, pre-service teachers, design, CT instruments

Validating a performance assessment of computational thinking for early childhood using item response theory

Despite growing interest in early childhood computational thinking (CT), there is a lack of validated assessments for children who are emerging readers. This paper presents validity and reliability evidence of a performance-based assessment of CT using item response theory (IRT) from 272 children aged 4–8. Using a two-parameter logistic model IRT model (2PL IRT), we confirmed that model- and item-level fits are acceptable. Item analyses revealed a high discriminability (M = 2.26, SD = 1.12) and a moderate item difficulty (M = −0.21; SD = 0.86), on average, across 19 items. Reliability analysis demonstrated that the assessment was substantially reliable (marginal reliability: rxx = 0.87). Differential item functioning (DIF) analyses indicated that the assessment estimated children's item parameters fairly, regardless of their gender and age. However, we confirmed gaps in latent ability (θ) of CT by gender and age: boys showed higher latent ability of CT than girls, and old children (above 72 months) showed higher latent ability than young children (below 72 months). Findings suggest the assessment is a fair measure that can serve as a reliable and valid tool to assess CT for children who are emerging readers.

A Systematic Umbrella Review on Computational Thinking Assessment in Higher Education

Computational Thinking (CT) is considered a core 21st century digital skill. The aspect of assessment is crucial and knowing what, who, when, how, and where to assess is important for assessment design. In this study, we conducted an umbrella review to gain insights regarding CT assessment in higher education. In total, we analyzed 11 reviews, focusing on: (1) bibliographical and methodological characteristics of the reviews; (2) aspects relevant of assessment design, including a) assessed constructs, b) applied assessment methodologies, and c) assessment contexts. Our findings suggest an increased attention on this topic. However, hardly any reviews reasoned the selection of their review methodology, and most of the reviews did not thoroughly examine existing reviews. Regarding assessment design aspects, most reviews did not confine their scope to higher education; however, findings on interventions and educational settings show commonalities. We identified 120 unique assessed constructs and around 10 types of assessment methods. Though a combined use of distinct assessment methods is suggested in reviews, guidelines for appropriate assessment design are yet to be constructed. Based on the findings, we argue that it is necessary to explore different combinations of assessment design in various contexts to construct assessment guidelines.

Computational Thinking Skill Level of Senior High School Students Majoring in Natural Science

Computational thinking (CT) is a skill integrated into various curricula in many countries. However, lack of student acceptance and limited assessment become challenges to integrating skill into the curriculum, specifically in developing countries like Indonesia. Therefore, this study aimed to validate the Indonesian version of the Computational Thinking Scale (CTS) and determine the CT skill level of high school students majoring in science. This study was conducted using a quantitative approach with a cross-sectional survey design. Participants were purposively selected based on certain criteria from a population of high schools in Yogyakarta, Indonesia. In this study, data were collected from 526 students with 19 items of CTS questionnaires and analysed using the Rasch model measurement. The findings showed that the adapted CTS met the fit criteria based on Rasch model measurement, except for one item. Based on the logit mean value of +1.69, the level of students' CT ability falls into the good category, where the most frequently implemented aspect is the problem-solving aspect, while the least frequently implemented aspect is algorithmic thinking. According to Differential Item Function analysis, there were differences in student responses based on coding experience. This study is expected to contribute to the field of CT assessment in science education. In addition, the results of this study can be an affirmation for educational policy makers in developing countries to integrate CT into the curriculum of natural science majors.

Exploring the multifaceted roles of mathematics learning in predicting students' computational thinking competency

BackgroundThere exist shared competencies between computational thinking (CT) and mathematics, and these two domains also mutually benefit from various teaching approaches. However, the linkages between mathematics and computational thinking lack robust empirical support, particularly from student-centered learning perspectives. Our study aimed to enhance our understanding of the connections between students' mathematics learning and computational thinking. To assess students' mathematics learning, we measured their beliefs about mathematics learning and their level of mathematical literacy (ML). Our hypothesis posited that students' beliefs concerning mathematics learning, encompassing their views on the nature of mathematics and their attitude towards the subject, can both directly and indirectly influence their CT, with ML serving as a mediating factor. Our data were gathered through surveys and tests administered to eighth- and ninth-grade students. Data were analyzed using partial least squares–structural equation modeling (PLS–SEM).ResultsThe evaluation of the measurement model indicated strong internal consistency for each construct. Both convergent and discriminant validity were also established. Upon assessing the structural model, it was found that beliefs about the nature of mathematics positively predicted attitudes towards mathematics, and this belief also indirectly predicted ML through positive attitudes towards mathematics. In addition, ML directly and positively predicted both CT subscales. Notably, a comprehensive mediating effect of ML on beliefs about mathematics learning and CT was identified in the analysis.ConclusionsThis study advances the understanding of the relationships between mathematics learning and CT. We have further confirmed the importance of mathematical literacy in predicting CT and its mediating role between beliefs about mathematics learning and CT. It is suggested that teachers could promote students’ CT competence by enhancing their mathematical literacy or integrating mathematics and CT into the same learning activities. Finally, we propose that upcoming investigations treat CT assessments as formative constructs, diverging from their reflective counterparts.

Correlation between High School Students’ Computational Thinking and Their Performance in STEM and Language Courses

According to numerous researchers, a clear and direct correlation exists between Computational Thinking (CT) and courses falling under the purview of Science, Technology, Engineering, and Mathematics (STEM), thereby advocating for the integration of CT into the curricula of STEM courses. Nonetheless, it is noteworthy that only a few studies have scrutinized this correlation in-depth. Most such studies connect the correlation tacitly and predominantly concentrate on the empirical assessment of CT within the curriculum of one STEM discipline. This research seeks to evaluate the Computational Thinking abilities of 80 high school students in Greece and discern the extent of correlation with their academic performance in STEM and Greek language courses. A longitudinal survey was executed to accomplish this objective, commencing with administering a test designed to gauge the fundamental components of Computational Thinking. It is worth noting that this test draws its inspiration from internationally recognized computer competitions and serves as a credible assessment tool. Subsequently, an assessment was carried out to ascertain the degree of correlation between students’ Computational Thinking aptitude and their written performance in the subjects encompassed by the STEM category and the Greek language courses. The outcomes of this investigation revealed the presence of a statistically significant correlation between students’ Computational Thinking proficiency and their performance in these academic subjects, further extending to the academic direction of study chosen by the students. Based on the findings of this research, implications and pedagogical recommendations are delineated while concurrently acknowledging the limitations encountered during this study.

Development and preliminary testing of the AlgoPaint Unplugged Computational Thinking Assessment for Preschool Education

The concept, development and assessment of computational thinking have increasingly become the focus of research in recent years. Most of this type of research focuses on older children or adults. Preschool age is a sensitive period when many skills develop intensively, so the development of computational thinking skills can already begin at this age. The increased interest in this field requires the development of appropriate assessments. Currently, there are only a limited number of computational thinking assessments for preschool children. Based on this shortcoming, an assessment tool, named AlgoPaint Unplugged Computational Thinking Assessment for Preschool, was created addressed for 4-7 years old children. It is a paperpencil-based test, which examines the following computational thinking domains: algorithms and debugging. Regarding computational concepts, simple instructions, simple and nested loops, and conditionals are included in the test. For the preliminary testing, AlgoPaint test was applied by 11 preschool teachers with 56 preschool age children. The test was also evaluated by 6 experts in algorithmic thinking working at universities. Based on the feedback given by the teachers and the experts, and the results of the children, AlgoPaint Computational Thinking Test was revised and completed. The revised version of the test is included in the appendix of the paper.

Embracing Computational Thinking as an Impetus for Artificial Intelligence in Integrated STEM Disciplines through Engineering and Technology Education 


 The scope and versatile nature of engineering and technology education as a discipline provide a platform for the integration of computational thinking (CT) into STEM education, accomplishing the goal of bringing not only computer science principles into the K-12 education but also the fundamentals of machine learning (ML) and artificial intelligence (AI) into the curriculum. Today, it is commonplace to say that artificial intelligence and machine learning technologies impact the workplace and continue to revolutionize as well as create new demands for solving daily world challenges. This article discusses the integration of computational thinking practices of decomposition, pattern recognition, algorithmic thinking, and abstraction as key to problem-solving practices that may enhance the development of AI and ML capabilities in high school students. The intent of this article is to contribute to ongoing discussions among educators, employers, parents, and all those concerned with how best to prepare a citizenry that is digitally revolutionized. Implications are offered for the assessment of CT integrated within STEM, curriculum, pedagogy, and professional development for STEM teachers.

Web Technologies in the Development of Computational Thinking of Students with Mental Disabilities

Computational thinking is an important and necessary part of a modern person’s thinking. It has been proven that the development of this way of thinking in students with mental disabilities allows them to navigate quickly in the modern world, identify problems and create complex solutions. Online schooling during the COVID-19 pandemic demonstrated the possibilities of the usage of web technologies in the education of children with mental disabilities. This study aims to evaluate the impact of web technologies on the development of computational thinking of students with mental disorders. The experiment involved 14 students aged 8-12 and 4 tutors. For 8 weeks children were trained in computational thinking and computer science. Assessment of computational thinking was performed with cCT-test by El-Hamamsi et al. before and after the experiment. After conducting computer science lessons using web technologies the respondents showed a higher level of computational thinking (M=15,7, SD=3,69), compared to the results of preliminary testing (M=5,93, SD=2,3). Web technologies can significantly increase the effectiveness of inclusive pedagogy, which establishes the importance of integrating web technologies into the teaching system in inclusive classes of general education schools.

Assessment of Computational Thinking: A Study of Preservice Teachers’ Knowledge and Beliefs

Aiming to bridge the gap in the field of computational thinking and computing education, this study examines preservice teachers’ knowledge of and self-efficacy related to CT assessment. It also develops a reliable instrument to understand the construct of teacher assessment of CT. Specifically, the research questions focus on investigating the extent of preservice teachers’ knowledge and level of self-efficacy of CT assessment, and identifying the difference between demographic variables on teacher knowledge and self-efficacy related to CT assessment. Adopting a cross-sectional survey design, the participants were 182 preservice teachers. Both descriptive statistics and Exploratory Factor Analysis were used to analyze the data. The results show that the preservice teachers know little about how to use CT assessment to help students, and they believe they know even less about using specific assessment techniques to accomplish assessment. Keywords: computational thinking, assessment, teacher knowledge, teacher self-efficacy

The Development of Computational Thinking and Mathematics Problem Solving Skills Through Mathematics Modelling Activities

The latest TIMSS report highlighted that secondary school students from Malaysia did not perform well in this global assessment on mathematics achievement. Some studies have pointed out that students’ mathematics achievement can be improved by equipping them with computational thinking. Therefore, this study examined the development of computational thinking skill and mathematics problem-solving skills via mathematics modelling activities among secondary students in Malaysia. The research instrument used for the assessment of computational thinking was adapted from UK Berbras Challenges and CAS Barefoot Team from United Kingdom. In addition, rubrics for development of mathematics problem solving skill was adapted from Polya problem solving model. This study adopted qualitative approach and the modelling activities were conducted for six weeks. The activities were introduction of modelling, individual modelling task and group modelling task. Data were also collected based on results of computational thinking test and mathematics test. Data from observation and interviews were analyzed, and the scores obtained from both tests were compared and analyzed as well. Based on the findings of the study, it could be concluded that the students progressed at different levels during the development of their computational thinking skill and mathematics problem-solving competency via mathematics modelling activities. These results indicated that mathematics modelling activities can be conducted at schools to improve students’ performance in Mathematics.

The Landscape of Computational Thinking Problems for Practice and Assessment

To provide practice and assessment of computational thinking, we need specific problems students can solve. There are many such problems, but they are hard to find. Learning environments and assessments often use only specific types of problems and thus do not cover computational thinking in its whole scope. We provide an extensive catalog of well-structured computational thinking problem sets together with a systematic encoding of their features. Based on this encoding, we propose a four-level taxonomy that provides an organization of a wide variety of problems. The catalog, taxonomy, and problem features are useful for content authors, designers of learning environments, and researchers studying computational thinking.

A systematic review of integrating computational thinking in early childhood education

Computational thinking education has become an increasingly popular topic among practitioners and researchers. However, rare is known how to effectively teach and learn computational thinking in early childhood education. To address this knowledge gap, this systematic review examined 26 studies on the teaching and learning of computational thinking in early childhood education from 2010 to 2022. The content knowledge, tools, pedagogical design, assessment methods, and learning outcomes were analyzed. Results indicated that, with age-appropriate instructional design, children could develop early concepts and skills of computational thinking, as well as other related skills such as communication, collaboration, and problem solving. Across the studies, we found that most studies used quantitative research methods, with direct assessment and observation being the most. Several challenges were identified: (1) achieving a deeper learning of computational thinking; (2) a lack of valid and reliable computational thinking assessments for children with a wider age range; (3) selecting appropriate learning tools; and (4) designing age-appropriate activities for young learners. Although with these challenges, computational thinking education could bring new learning opportunities and enhance children's computational thinking skills, as well as other non-cognitive skills such as critical thinking, body-material interaction, and hand-eye coordination. This systematic review informs future endeavors in theorizing a digital learning framework that can integrate computational thinking into early childhood education.

Insights into computational thinking from think‐aloud interviews: A systematic review

AbstractThis systematic review examines 35 empirical studies featuring the use of think‐aloud interviews in computational thinking (CT) research. Findings show that think‐aloud interviews (1) are typically conducted in Computer Science classrooms and with K‐12 students; (2) are usually combined with other exploratory CT assessment tools; (3) have the potential to benefit learners with special needs and identify the competency gaps through involving diverse participants; (4) are conducted in the absence of cognitive models and standard procedures; and (5) display insufficient definitional and methodological rigor. Theoretically, this review presents a systematic assessment about the application of think‐aloud interviews in CT studies and identifies the limitations in existing CT‐related think‐aloud studies. Practically, this review serves as a reference for studying the cognitive processes during CT problem‐solving and provides suggestions for CT researchers who intend to incorporate think‐aloud interviews in their studies.

Comparing the psychometric properties of two primary school Computational Thinking (CT) assessments for grades 3 and 4: The Beginners' CT test (BCTt) and the competent CT test (cCTt).

With the increasing amount of research around Computational Thinking (CT) and endeavors introducing CT into curricula worldwide, assessing CT at all levels of formal education is of utmost importance to ensure that CT-related learning objectives are met. This has contributed to a progressive increase in the number of validated and reliable CT assessments for K-12, including primary school. Researchers and practitioners are thus required to choose among multiple instruments, often overlapping in their age validity. In this study, we compare the psychometric properties of two of these instruments: the Beginners' CT test (BCTt), developed for grades 1-6, and the competent CT test (cCTt), validated for grades 3-4. Classical Test Theory and Item Response Theory (IRT) were employed on data acquired from 575 students in grades 3-4 to compare the properties of the two instruments and refine the limits of their validity. The findings (i) establish the detailed psychometric properties of the BCTt in grades 3-4 for the first time, and (ii) through a comparison with students from the same country, indicate that the cCTt should be preferred for grades 3-4 as the cCTt is able to discriminate between students of low and medium ability. Conversely, while the BCTt, which is easier, shows a ceiling effect, it is better suited to discriminate between students in the low ability range. For these grades, the BCTt can thus be employed as a screening mechanism to identify low ability students. In addition to providing recomendations for use of these instruments, the findings highlight the importance of comparing the psychometric properties of existing assessments, so that researchers and practitioners, including teachers and policy makers involved in digital education curricular reforms, may take informed decisions when selecting assessments.

Assessing Algorithmic Thinking Skills in Relation to Age in Early Childhood STEM Education

In the modern digital era, intensive efforts are made to inject computational thinking (CT) across science, technology, engineering, and mathematics (STEM) fields, aiming at formulating a well-trained citizenry and workforce capable of confronting intricate problems that would not be solvable unless exercising CT skills. Focusing on contributing to the research area of CT assessment in the first two years of primary school, we investigated the correlation of algorithmic thinking skills, as a fundamental CT competency, with students’ age in early childhood settings. This article reports a relevant research study, which we implemented under the umbrella of quantitative methodology, employing an innovative assessment tool we constructed for serving the needs of our study. The research was conducted within the context of the environmental study course, adding to the efforts of infusing CT into STEM fields. The study results shed light on the correlation between algorithmic thinking skills and age in early childhood, revealing that age is a predictor factor for algorithmic thinking and, therefore, for CT.

The Competent Computational Thinking Test: Development and Validation of an Unplugged Computational Thinking Test for Upper Primary School

With the increasing importance of Computational Thinking (CT) at all levels of education, it is essential to have valid and reliable assessments. Currently, there is a lack of such assessments in upper primary school. That is why we present the development and validation of the competent CT test (cCTt), an unplugged CT test targeting 7–9 year-old students. In the first phase, 37 experts evaluated the validity of the cCTt through a survey and focus group. In the second phase, the test was administered to 1519 students. We employed Classical Test Theory, Item Response Theory, and Confirmatory Factor Analysis to assess the instruments’ psychometric properties. The expert evaluation indicates that the cCTt shows good face, construct, and content validity. Furthermore, the psychometric analysis of the student data demonstrates adequate reliability, difficulty, and discriminability for the target age groups. Finally, shortened variants of the test are established through Confirmatory Factor Analysis. To conclude, the proposed cCTt is a valid and reliable instrument, for use by researchers and educators alike, which expands the portfolio of validated CT assessments across compulsory education. Future assessments looking at capturing CT in a more exhaustive manner might consider combining the cCTt with other forms of assessments.

Robot programming versus block play in early childhood education: Effects on computational thinking, sequencing ability, and self‐regulation

AbstractProgrammable robotics is recently used in early childhood education (ECE) to introduce programming and computational thinking (CT) skills. However, there is a further need for research to contrast the efficacy of children's participation in robot programming and traditionally beneficial ECE activities. The present study thus investigated the effects of a robot programming intervention versus a block play program on kindergarteners' CT, sequencing ability, and self‐regulation. The experiment (robot programming) versus comparison (block play) condition was randomly assigned to four kindergarten classes, which included 101 kindergarteners (M = 64.78 months, SD = 7.64). Statistical analyses revealed that the robot programming group (N = 54) had experienced greater gains over time in sequencing ability relative to those in the block play group (N = 47; F = 5.09, p < 0.05). Children in the robot programming group with lower level of self‐regulation at baseline showed larger improvements in sequencing ability over time relative to the block play group (F = 2.37, p = 0.01). Also, children in the robot programming group with older age showed larger improvements in CT over time relative to the block play group (F = 2.40, p < 0.01). The study demonstrates the positive benefits of robot programming to early childhood development in terms of CT and sequencing ability, compared to a traditional curriculum activity in ECE—block play. Practitioner notesWhat is already known about this topic Screen‐free robot programming can enhance preschool children's computational thinking (CT). Block play can enhance preschool children's mathematics and executive functioning. Both robot programming and block play are engaging for preschool children. What this paper adds An unplugged CT assessment is used to measure and compare the effects of both robot programming and block play interventions among preschool children. Robot programming outperforms block play in promoting children's school readiness skills such as sequencing ability. Children with lower self‐regulation skills benefited more from the robot programming intervention. Implications for practice and/or policy Robot programming and CT education should be expanded in diverse early childhood settings to boost the positive effects. Technology‐enhanced curricula should be integrated into early childhood education. Teachers should receive training on robot programming in addition to more traditional skills such as scaffolding children's block play.