Assessment Of Higher Order Thinking Skills Research Articles

Comparison between MCQ, Extended matching questions (EMQ) and Script concordance test (SCT) for assessment among first-year medical students - A pilot study.

Single-best response types of MCQs are commonly used tools in medical assessment. However, these are not particularly apt for the assessment of higher-order thinking skills (HOTS) among students. Assessment of HOTS and clinical reasoning skills requires unexplored tools like extended matching questions (EMQs) and SCTs. The aim is to assess HOTS and clinical reasoning skills during formative assessment among medical students post a simulation-based education (SBE) using EMQ and SCT on the topic of shock and collect student perceptions regarding new assessment tools. The research is an observational descriptive study. Fifty-two first-year medical students were asked to take a formative assessment consisting of 20 MCQs, 6 EMQs, and 2 SCT post a SBE during July 2022 on the topic of shock. MCQs were categorized into themes of aetiology, pathophysiology, and management of shock. These categorized MCQs were compared and analyzed with EMQs and SCTs prepared on the same themes. The data analysis by a one-way analysis of variance (ANOVA) was conducted to compare the difference in per cent mean scores of MCQ with EMQ and MCQ with SCT. The overall mean scores and also under the theme of aetiology and management, the student's score in MCQ were higher than EMQ and SCT and the difference was statistically significant with the P value (≤0.001). Students scored better in familiar assessment tools like MCQ, but majority of the students were of the opinion that EMQ tested the delivered content better and SCT tested the clinical application better.

Assessment of Higher-order Thinking Skills in Conversation Situation

Assessment of students’ higher-order thinking skills is a key issue in education, and data-driven multimodal learning analytics is the way forward. Mining the essential content of collaborative conversations is the key to the assessment of higher-order thinking skills, and the integration of learning situations is crucial. In this paper, we provide digital collaborative learning situations through a virtual simulation platform. Based on the conversation data, decision-making behaviour data and environmental data generated in the collaborative learning process, a set of data-driven higher-order thinking skills identification methods integrating situation and conversation are designed. The method includes four steps: situation identification and modelling, text extraction, situational semantic analysis, and semantic flow-based modelling. Compared with traditional methods such as questionnaires and interviews, the ability identification driven by conversation data realizes the direct representation of higher-order thinking skills, which has real-time performance and higher reliability. What’s more, integrating situational information into the learning analysis model improves the accuracy of ability identification results from the pragmatic level.

Chemistry Teacher’s Perception About Higher Order Thinking Skills Assessment

Higher order thinking skills are an assessment that must be integrated into 2013 Curriculum learning process. The purpose of this study is to investigate the chemistry teacher’s perception about higher order thinking skills assessment. Descriptive qualitative was used in this study. This study implicated 10 chemistry teacher from 10 senior high school in West Kalimantan. Data on chemistry teacher’s perception about higher-order thinking skills assessment were obtained from questionnaire consist of 10 questions. The results showed that most of the chemistry teachers in West Kalimantan already know and implement higher order thinking skills assessment in learning, like final exam and daily test. The HOTS assessment has been implemented by most teachers, but the implementation was not optimal even though in terms of knowledge all teachers already know about the HOTS assessment.

Developing the MathSci 21st app: Enhancing higher-order thinking skills assessment in mathematics and science education within an Islamic context

The innovation of digital assessment holds profound potential for enhancing educational quality. To measure higher-order thinking skills, intrinsic to effective problem-solving in science and mathematics education, and to cultivate digital literacy, a specialized platform is imperative. This study delineates the developmental trajectory of the MathSci 21st app, designed to assess mathematics and science proficiency within the Islamic context. Emphasizing the pivotal role of higher-level thinking skills in the contemporary landscape, the research method unfolds across distinct phases: Akker (preliminary research), prototype stage (prototyping), summative evaluation, and systematic reflection and documentation. This article confines its focus to preliminary research and prototype stages. The validation of the application prototype engaged a panel of ten experts, while a controlled trial encompassed 32 high school students and one educator. Utilizing observation sheets, questionnaires, and tests as research tools, comprehensive data analysis was executed employing both quantitative and qualitative methods. Research outcomes affirm the feasibility of the Prototype MathSci 21st app, an Android-based competency assessment tool characterized by its integrated and contextual dimensions. Android-based applications not only heighten efficiency and efficacy but also exhibit environmental conscientiousness by reducing paper usage. Additionally, their user familiarity augments acceptability. Significantly, the MathSci 21st app expedites assessment, empowering educators to provide prompt feedback and expedite future learning analysis. This study pioneers a digital assessment paradigm tailored to intricate higher-order thinking skills, thereby addressing critical educational imperatives in mathematics and science within the Islamic milieu.

Development and validation of a higher-order thinking skills assessment scale for pre-service teachers

Higher-order thinking skills (HOTS) are highly required to be possessed by prospective teachers in the 21st century but there are limited studies on the development of appropriate measurement tools for this HOTS. Therefore, this study was conducted to develop a HOTS quantitative assessment instrument for pre-service teachers by collecting data from a sample of 701 pre-service teachers in China. The EFA (n = 310) confirmed a "4-dimensional, 14-factor" assessment structure which includes four main dimensions:critical thinking, problem-solving, teamwork, and practical innovation skills. Moreover, the correlated and hierarchical models run by CFA (n = 392) were consistent with the measurement theory structural assumptions, and the scale was observed to have ideal values of internal consistency reliability, convergent validity, and discriminant validity. The findings suggest that the PT-HOTS Test Scale is reliable and valid to measure the level of HOTS possessed by preservice teachers.

Higher-order thinking skills assessment in 3D virtual learning environments using motifs and expert data

The research reported in this paper addresses the problem of assessing higher-order thinking skills, such as reflective and creative thinking, within the context of virtual learning environments. Assessment of these skills requires process-based observations and evaluation, as the output-based methods have been found to be insufficient. Virtual learning environments offer a wealth of data on the process, which makes them good candidates for process-based evaluation, but the existing assessment methods in these environments have shortcomings, such as reliance on large data sets, inability to offer specific feedback on actions, and the lack of consideration for how actions are integrated into bigger tasks. Demonstrating and confirming the ability of three-dimensional virtual learning environments to work with process metrics for assessment, we propose and evaluate the use of motifs as an assessment tool. Motifs are short and meaningful combination of metrics. Combining time-ordered motifs with a similarity analysis between expert and learner data, our proposed approach can potentially offer feedback on specific actions that the learner takes, as opposed to single output-based feedback. It can do so without the use of large training datasets due to reliance on expert data and similarity analysis. Through a user study, we found out that such a motif-based approach can be effective in the assessment of higher-order thinking skills while addressing the identified shortcomings of previous work. We also address the limited research on similarity-based analysis methods, compare their effectiveness, and show that utilizing different similarity measures for different tasks may be a more effective approach. Our proposed method facilitates and encourages the involvement of instructors and course designers through the definition of motifs and expert problem-solving paths.

The effectiveness of a proposed strategy for teaching Geography through artificial intelligence applications in developing secondary school students’ higher-order thinking skills and achievement

The study investigated the effectiveness of employing a proposed strategy for teaching geography using artificial intelligence applications in developing higher-order thinking skills and achievement among secondary school. The quasi-experimental method was used in this study. An observation card for higher-order thinking skills and an achievement test in the geography subject were used. The sample consisted of (60 female students) selected from the second year of secondary school students. The study showed significant differences at (0.05) between the mean scores of the students in the control and experimental groups on the achievement test in favor of the experimental group in the posttest. Also, differences were shown between the assessment means of the students in the control and experimental groups on the higher-order thinking skills assessment card in favor of the experimental group. The effect size came high. The study recommends the need to emphasize the expansion of the use of artificial intelligence applications in teaching geography.

The Validity of an Instrument The Assessment of Higher Order Thinking Skills About Metabolism

Background: The test results of the higher-order thinking skills assessment instrument of students at several high schools in West Sumatra obtained an average score of 17.88-35.19 which is still low. One way to overcome students' low level of high-order thinking skills is to develop an instrument for assessing higher-order thinking skills. This study aims to produce a valid, practical, and good quality item assessment instrument of students' high order thinking about metabolic material for class XII SMA / MA. Methods: This study uses a 4-D development model. The subjects of this study were 3 validators and 27 students of class XII SMA. The data collection instrument was a validity questionnaire and a practicality test. Results: The data obtained are logical validation with an average percentage of 84.90% valid criteria, empirical validity with a correlation between 0.27-0.54 with valid criteria. Has an average percentage on practical criteria of 86.64%, reliability 0.83 with high categories Conclusions: Instruments for assessing the ability to think at a high level about metabolic material are logically valid and empirically valid, practical, reliable, moderate difficulty level, distinguishing power and good quality options.

Penyusunan soal tes bermuatan lower dan higher-order thinking skills pada guru SMP

Training for Junior High School’s teacher on constructing LOTS and HOTS questions aims to (1)improve teacher competence on designing lower and higher-order thinking skills assessment; and (2)provide teachers with an understanding of the formulation of successful test questions and the evaluation criteria set by the Ministry of Education and Culture. The training was given to 31 teachers of SMP IT Cendikia Takengon in stages; (1) pre-training observations; (2) tutorial and practice; (3) assistance; and (4) evaluation. The training conducted on August 14 to October 20, 2020. Based on the results of evaluation and reflection, it is found that; (1) teachers were able to write questions, both multiple-choice and essays questions, following the principles of writing questions; (2) teachers were able to understand and distinguish between cognitive levels (levels 1,2 and 3) and cognitive dimension (C1, C2, C3, C4, C5, and C6); (4) teachers generally have been able to compile HOTS questions although some teachers still have difficulty designing HOTS questions; (5) Teachers understand that HOTS questions are not difficult questions, but level 3 questions that involve reasoning; and (6) teachers understand that structured and non-structured test if properly designed have the potential to measure both lower and higher-order thinking skills.

Developing Higher Order Thinking Skills Assesment Instruments on Invironmental Pollution Material for Class VII

This study aim to produce a valid and practical assessment instrument for higher order thinking skills on environmental pollution material at the C4-C6 cognitive level. The background of this research is that the questions made by the teacher are still at the C1-C3 level and there are still obstacles for teachers in developing higher order thinking skills assessment instruments, while in the 2013 curriculum teachers are required to be able to provide questions in the C4-C6 domain. This type of research is research development (Research and Development) which is a research method used to produce certain products. This type of research uses 3 stages of the 4-D model, namely the define, design and develop stages.Based on the research conducted, a very valid assessment instrument of 3,32 was produced by the validator, the empirical validity there were 30 valid questions and 10 invalid questions, the reliability in this study was 0,77 with the category of reliability, 94,79% teacher practicality and practicality. students amounted to 80,80%, the difficulty level of the questions was between 0,31 to 0,70 with moderate criteria and the question difference had sufficient criteria. This research can be used as a reference for other researchers related to the development of higher order thinking skills assessment instruments.

Developing assessment of higher order thinking skills in physics learning based on local wisdom

This study aims to develop an assessment of higher-order thinking skills in physics-based on local wisdom. This research through four (4) stages of development: defining, designing, developing, and distributing. The results show that the validity value of 3.94 on the rating scale of 3.0 ≤ SV ≤4.0 with a correct category and can be implemented with small revisions. The practicality of assessment is obtained from the readability of assessment instruments with a percentage of 81%, the level of difficulty of the instrument with a percentage of 72%, and responses from respondents with a percentage of 83%. Assessment effectiveness is obtained from learning completeness, where classical completeness is 100%, and indicator completeness is 90.5%. Thus the assessment of higher-order thinking skills in physics-based on local wisdom can be said to be feasible to be disseminated.

Pengembangan Instrumen Penilaian Kemampuan Berpikir Tingkat Tinggi Materi Sistem Pendengaran dan Sonar SMP

The aim of this study is to produce a higher-order thinking skills assessment instrument on the material of the hearing and sonar system for Junior high school students that are valid and reliable. Indicators of higher-order thinking skills (IKBTT) are developed based on the cognitive dimension of Bloom's Taxonomy which includes the skills to analyze (C4), evaluate (C5) and create (C6). The assessment instrument developed was a multiple choice test with open reasons. Multiple choice test with open reason is a multiple choice question with four answer choices completed with the reason for choosing an answer. The instrument is equipped with an assessment rubric in the form of a holistic rubric to assess the relationship between multiple choice answers and reasons for choosing answers. The results of the validation of material aspects, construction, and language show that the products developed are very feasible with an average score of 85.7%. The instrument was empirically tested with the number of subjects trying as many as 59 students of 8th grade Junior high school. Based on the results of empirical trials of multiple choice questions with open reasons obtained 25 valid questions with multiple choice reliability of 0.745, and reason reliability of 0.929. The number of questions produced based on C4 cognitive dimensions were 12 questions (48%), C5 cognitive dimensions were 9 questions (36%), and C6 cognitive dimensions were 4 questions (16%). This study produced an instrument for assessing higher-order thinking skills in the material of the hearing and sonar system for Junior high school students that were valid and reliable.

Computer-based assessment of mathematics into the twenty-first century: pressures and tensions

In recent decades, technology has influenced various aspects of assessment in mathematics education: (1) supporting the assessment of higher-order thinking skills in mathematics, (2) representing authentic problems from the world around us to use and apply mathematical knowledge and skills, and (3) making the delivery of tests and the analysis of results through psychometric analysis more sophisticated. We argue that these developments are not pushing mathematics education in the same direction, however, which creates tensions. Mathematics education—so essential for educating young people to be creative and problem solving agents in the twenty-first century—is at risk of focusing too much on assessment of lower order goals, such as the reproduction of procedural, calculation based, knowledge and skills. While there is an availability of an increasing amount of sophisticated technology, the related advances in measurement, creation and delivery of automated assessments of mathematics are however being based on sequences of atomised test items. In this article several aspects of the use of technology in the assessment of mathematics education are exemplified and discussed, including in relation to the aforementioned tension. A way forward is suggested by the introduction of a framework for the categorisation of mathematical problem situations with an increasing sophistication of representing the problem situation using various aspects of technology. The framework could be used to reflect on and discuss mathematical assessment tasks, especially in relation to twenty-first century skills.

Higher-Order Thinking Development through Adaptive Problem-based Learning

In this paper we propose an approach to organizing Adaptive Problem-Based Learning (PBL) leading to the development of Higher-Order Thinking (HOT) skills and collaborative skills in students. Adaptability of PBL is expressed by changes in fixed instructor assessments caused by the dynamics of developing HOT skills needed for problem solving, flexible choice of control tests and problems for students, and adaptive formation of HOT skills within heterogeneous collaborative groups. It induces the students to develop HOT skills and collaborative skills through a combination of personalized and collaborative PBL. Adaptability of PBL is realized by taking into account values of the proposed coefficient of HOT skills development. The two-stage process of adaptive PBL allows guided development of HOT skills in students during study of a subject. Attention in the first stage is devoted to development of analytical HOT skills in students through personalized PBL. The main attention on the second stage is devoted to the development of creative HOT skills and collaborative skills in students through collaborative PBL. The proposed approach provides effective development of HOT skills and collaborative skills in students owing to: availability of a two-stage adaptive PBL process, complex and adaptive assessment of HOT skills, dynamic choice of control tests and problems for students, adaptive formation of HOT skills heterogeneous collaborative groups, and management of HOT skills development of students.