Higher-order Cognitive Skills Research Articles

Undergraduate Biology Lecture Courses Predominantly Test Facts about Science Rather than Scientific Practices.

Scientific practices are the skills used to develop scientific knowledge and are essential for careers in science. Despite calls from education and government agencies to cultivate scientific practices, there remains little evidence of how often students are asked to apply them in undergraduate courses. We analyzed exams from biology courses at 100 institutions across the United States and found that only 7% of exam questions addressed a scientific practice and that 32% of biology exams did not test any scientific practices. The low occurrence of scientific practices on exams signals that undergraduate courses may not be integrating foundational scientific skills throughout their curriculum in the manner envisioned by recent national frameworks. Although there were few scientific practices overall, their close association with higher-order cognitive skills suggests that scientific practices represent a primary means to help students develop critical thinking skills and highlights the importance of incorporating a greater degree of scientific practices into undergraduate lecture courses and exams.

Tools that Eliminate Gender Differences in the Academic Self-Concept

Despite critical thinking supporting a deeper understanding of the scientific process, university activities prioritize lower cognitive processes, such as remembering skills. Also, it is unclear whether gender biases in interest toward science exist in university science degrees. These hinder students from achieving their goals effectively since students’ test scores and motivation improve with project-based learning. The main goal of this study is to examine the influence of active methodology based on research project-based learning (RPBL) on students’ overall perception and its variation across gender groups in relation to the acquisition of higher-order cognitive skills. The RPBL activity will consist of a straightforward microbial ecology project in which the students will use conventional and affordable lab equipment. The project will address all higher-order thinking skills levels included in Bloom’s taxonomy. We evaluated students’ perception of their learning outcomes on lab and cognitive skills, including the effect of gender, using two online surveys we passed before and after the RPBL activity. The results displayed that the students’ opinion regarding the project was decidedly favorable. Most of the students view the potential effort required for these activities positively, as it pertains to the subject matter and enhances their learning. However, the gender differences that were observed prior to the project’s implementation disappeared once the activity was carried out. Female students lacked confidence in their statistical skills while they had high confidence in their laboratory skills compared to male students. After attending our RPBL activity, female students gained confidence in statistics, and male students gained confidence in laboratory skills. This project reveals that a straightforward and affordable RPBL activity, which would not suppose a substantial additional workload for university lecturers, holds the potential to serve as a valuable tool to eliminate gender segregation, enhancing students’ academic self-concept, and improving their prospects for success. Additionally, it emphasizes higher cognitive skills, particularly the critical thinking skills that are essential within the university environment.

Blended learning in nursing pharmacology: elevating cognitive skills, engagement and academic outcomes.

Pharmacological education is crucial for healthcare professionals to safely manage medications and reduce errors. Traditional lecture-based learning (LBL) often struggles to address this complexity, whereas newer methods, such as flipped classrooms and problem-based learning, yield mixed results, particularly in pre-clinical contexts, owing to students' limited experience. Our nursing pharmacology course under LBL recorded a high failure rate of 37.8% and marginal passing scores across five cohorts (n = 849 students). An analysis using Bloom's taxonomy revealed significant gaps in higher-order cognitive skills. As a remedy, the course was transformed into a novel blended learning format that integrated question-based learning (QBL) to enhance critical thinking across all cognitive levels. This model blends asynchronous and synchronous learning, is tailored to individual needs in large classes, and fosters continuous, student-centric learning. The redesign markedly decreased the failure rate by approximately 2.8-fold and increased the average grade by 11.8 points among 426 students. It notably improved the pass rates in advanced cognitive categories, such as "Evaluate" and "Create" by 19.0% and 24.2%, respectively. Additionally, the blended course showed increased student engagement, reflecting a dynamic and effective learning environment that significantly elevated participation and academic outcomes at all cognitive levels. This study demonstrated the profound impact of blended learning in pharmacology. By integrating QBL with various teaching methods, it surpasses traditional lecture-based limitations, enhancing engagement and understanding of complex topics by nursing students. Notable improvements in foundational and advanced learning suggest its broader application in health professionals' education, effectively equipping students for clinical pharmacology challenges.

Postexamination item analysis of undergraduate pediatric multiple-choice questions exam: implications for developing a validated question Bank.

Item analysis (IA) is widely used to assess the quality of multiple-choice questions (MCQs). The objective of this study was to perform a comprehensive quantitative and qualitative item analysis of two types of MCQs: single best answer (SBA) and extended matching questions (EMQs) currently in use in the Final Pediatrics undergraduate exam. A descriptive cross-sectional study was conducted. We analyzed 42 SBA and 4 EMQ administered to 247 fifth-year medical students. The exam was held at the Pediatrics Department, Qena Faculty of Medicine, Egypt, in the 2020-2021 academic year. Quantitative item analysis included item difficulty (P), discrimination (D), distractor efficiency (DE), and test reliability. Qualitative item analysis included evaluation of the levels of cognitive skills and conformity of test items with item writing guidelines. The mean score was 55.04 ± 9.8 out of 81. Approximately 76.2% of SBA items assessed low cognitive skills, and 75% of EMQ items assessed higher-order cognitive skills. The proportions of items with an acceptable range of difficulty (0.3-0.7) on the SBA and EMQ were 23.80 and 16.67%, respectively. The proportions of SBA and EMQ with acceptable ranges of discrimination (> 0.2) were 83.3 and 75%, respectively. The reliability coefficient (KR20) of the test was 0.84. Our study will help medical teachers identify the quality of SBA and EMQ, which should be included to develop a validated question bank, as well as questions that need revision and remediation for subsequent use.

Exploring graduate students’ perception and adoption of AI chatbots in Zimbabwe: Balancing pedagogical innovation and development of higher-order cognitive skills

Exploring graduate students’ perception and adoption of AI chatbots in Zimbabwe: Balancing pedagogical innovation and development of higher-order cognitive skills

Interprofessional Differences in Multidimensional Self-Efficacy Associated With Professional Performance in Nephrology During Case-Based Learning

IntroductionPostgraduate medical education assumes rising importance in the rapidly advancing field of medicine. Case-based learning (CBL), a learner-centered pedagogy employing clinical cases to improve decision-making, is widely embraced in postgraduate medical education, including nephrology. Studies suggested that learning self-efficacy (SE) was tightly associated with learning motivation and academic performance, but very few examined this association in nephrology postgraduate education. None evaluated whether there were interprofessional differences concerning such association. MethodsIn 2022, we prospectively enrolled physicians and nurses participating in chronic kidney disease (CKD) care from institutions around Taiwan. They completed the Professional Medical Learning Self-efficacy questionnaire after attending >1 CBL session involving CKD care. We undertook confirmatory factor analysis, followed by structural equation modeling to evaluate associations between five dimensions of learning SE (conceptual understanding (CU), higher-order cognitive skills (HC), practical work (PW), everyday application (EA), and medical science communication (MSC)) and their professional SE in nephrology according to medical professions. Results513 healthcare providers were surveyed. The convergent and construct validity of our questionnaire were satisfied after analyses. We found that better perceived professional performance in the form of higher professional SE in nephrology was significantly associated with all five dimensions of learning SE among physicians and nurses. Only CU and PW were significantly associated with physicians' professional performance, whereas among nurses, only HC and MSC were significantly associated. ConclusionWe showed that learning SE was an important determinant of nephrology professional performance. Different medical professions pose influences on major SE dimensions.

Desempenho do ChatGPT nas questões da avaliação anual de residentes do Colégio Brasileiro de Radiologia

Abstract Objective: To test the performance of ChatGPT on radiology questions formulated by the Colégio Brasileiro de Radiologia (CBR, Brazilian College of Radiology), evaluating its failures and successes. Materials and Methods: 165 questions from the CBR annual resident assessment (2018, 2019, and 2022) were presented to ChatGPT. For statistical analysis, the questions were divided by the type of cognitive skills assessed (lower or higher order), by topic (physics or clinical), by subspecialty, by style (description of a clinical finding or sign, clinical management of a case, application of a concept, calculation/classification of findings, correlations between diseases, or anatomy), and by target academic year (all, second/third year, or third year only). Results: ChatGPT answered 88 (53.3%) of the questions correctly. It performed significantly better on the questions assessing lower-order cognitive skills than on those assessing higher-order cognitive skills, providing the correct answer on 38 (64.4%) of 59 questions and on only 50 (47.2%) of 106 questions, respectively (p = 0.01). The accuracy rate was significantly higher for physics questions than for clinical questions, correct answers being provided for 18 (90.0%) of 20 physics questions and for 70 (48.3%) of 145 clinical questions (p = 0.02). There was no significant difference in performance among the subspecialties or among the academic years (p > 0.05). Conclusion: Even without dedicated training in this field, ChatGPT demonstrates reasonable performance, albeit still insufficient for approval, on radiology questions formulated by the CBR.

Exploring graduate students’ perception and adoption of AI chatbots in Zimbabwe: Balancing pedagogical innovation and development of higher-order cognitive skills

Exploring graduate students’ perception and adoption of AI chatbots in Zimbabwe: Balancing pedagogical innovation and development of higher-order cognitive skills

Examining the Validity and Reliability of ChatGPT 3.5-Generated Reading Comprehension Questions for Academic Texts

This research examines the capacity of ChatGPT 3.5 in generating reading comprehension questions for academic texts, with a focus on their alignment with higher-order cognitive skills as per Bloom’s Taxonomy. A paper-based test comprising 30 multiple-choice questions was constructed using ChatGPT 3.5, based on three selected TOEFL ITP reading comprehension passages. The study employed a mixed-methods approach, integrating qualitative content analysis to assess the cognitive level of each question and quantitative methods to analyze student responses. Data collection involved administering the AI-generated questions to students and scoring their responses. Analysis techniques included Pearson correlation coefficients to determine validity and reliability analysis using Cronbach's Alpha to measure internal consistency. The findings revealed that ChatGPT 3.5 is capable of producing questions that cover a range of cognitive levels, from analysis to creation, however only 10 out of 30 questions met the validity criteria, indicating a need for improvement in the AI's question generation process. The reliability of these questions was moderate, suggesting a reasonable level of internal consistency. The study concludes that while AI-generated questions show promise in educational assessments, ongoing improvement of AI models is necessary to enhance their effectiveness. The implications of this research are significant for the future integration of AI in educational settings, indicating a potential role for AI in developing meaningful assessment tools. The study recommends future research to explore various question types and incorporate student feedback to optimize the effectiveness of AI in education.

Developing a taxonomy of hazards and hazard mitigation strategies for Motorised mobility scooter users

The increased use of motorised mobility scooters (MMSs) presents a road safety challenge as using a MMS has risks for the user, pedestrians, and other road users. In relation to enhancing MMS driving safety, much of the training and available literature focuses on training vehicular control. Equally important is the need to investigate higher-order cognitive skills involved in driving MMSs, particularly hazard perception. Through a large questionnaire study with MMS users, we develop a taxonomy of the types of hazard MMS users encounter when crossing roads and strategies that are used to negotiate these hazards. Whilst MMS experience modulated hazard perception and strategy use, a core set of hazards and strategies were identified that have policy and practice implications for training interventions and the built environment. Exploration of the advantages and disadvantages of MMS use indicated its impact on various wellbeing outcomes as well as some potential barriers to use.

Evaluating Experimental Activities in Czech Chemistry Textbooks: A Critical Analysis

Abstract This study analysed the nature and integration of experimental activities in Czech lower-secondary chemistry textbooks which are currently in use. Focusing on four dominant textbook series (published since 1990’s), and one recently (2019) published and certified, it investigated the offered types of student activities, their cognitive demands, placement in the educational process, and inquiry levels. The findings reveal two distinct groups of textbooks based on the quantity of experimental activities. Some textbooks align with traditional teaching methods, emphasising teacher-led demonstrations, while others attempt a balance between safety concerns and student engagement. However, a general lack of focus on higher-order cognitive skills and inadequate scaffolding for scientific process skills development was found. The results showed future research should investigate the impact of experimental activities on student outcomes, highlighting the need for more modern approaches in chemistry education.

Exploring differences in self-regulated learning strategy use between high- and low-performing students in introductory programming: An analysis of eye-tracking and retrospective think-aloud data from program comprehension

Previous studies have reported mixed results regarding the relationship between students’ use of self-regulated learning (SRL) strategies and their performance in introductory programming courses. These studies were constrained by their reliance on self-report questionnaires as a means of collecting and analysing data. To address this limitation, this study aimed to employ eye-tracking and retrospective think-aloud techniques to identify differences in SRL strategy use for program comprehension tasks between high-performing students (N = 31) and low-performing students (N = 31) in an undergraduate programming course. All participants attended individual eye-tracking sessions to comprehend two Python program codes with different constructs. Their eye-tracking data and video-recalled retrospective think-aloud data were captured and recorded for analysis. The findings reveal that higher-order cognitive skills, such as elaboration and critical thinking, were mostly adopted by high-performing students, while basic cognitive and resource management strategy, such as rehearsal and help-seeking, were mostly employed by low-performing students when comprehending the program codes. This study not only demonstrates the design of combining eye-tracking and retrospective think-aloud data to explore students’ use of SRL strategies but also provides evidence to support the notion that program comprehension is a complex process that cannot be effectively addressed by employing merely rudimentary strategies, such as repetitively reading the same code segment. In the future, researchers could explore the possibility of using a webcam to monitor and assess students’ online programming processes and provide feedback based on their eye movements. They could also examine the effects of SRL strategies training on students’ motivation, engagement, and performance in various types of programming activities.

Examining the Impact of Bloom's Taxonomy Integration in Entrepreneurship Education on the Competitiveness and Performance of Micro, Small, and Medium Enterprises (MSMEs)

This book review examines the impact of integrating Bloom's Taxonomy into entrepreneurship education on the competitiveness and performance of Micro, Small, and Medium Enterprises (MSMEs). Through a comprehensive analysis of literature, empirical evidence, and expert opinions, the review highlights the significance of this pedagogical approach in fostering critical thinking, problem-solving, and innovation skills among entrepreneurship students. Empirical studies demonstrate improvements in students' competencies and learning outcomes following exposure to Bloom's Taxonomy-aligned curricula. Case studies, such as the Entrepreneurship Development Institute of India (EDII), provide evidence of the practical relevance of this approach in addressing challenges faced by MSMEs. Expert opinions underscore the importance of preparing students for the dynamic demands of the entrepreneurial ecosystem through curriculum designs that prioritize higher-order cognitive skills development. Overall, the review emphasizes the transformative potential of Bloom's Taxonomy integration in shaping the future of entrepreneurship education and enhancing the competitiveness of MSMEs in global markets.

Effectiveness of the CinQASE Web Integrated Learning Model to Train Higher Order Thinking Skills

The lack of training in higher-order cognitive thinking skills in physics makes most students of class XI MIPA in a high school in Marabahan unable to solve problems that require the ability to analyze (C4) and evaluate (C5) well. This study was intended to describe the effectiveness of using the web-integrated CinQASE learning model to train higher-order thinking skills in the C4 and C5 domains. This study used the pre-experimental one-group pretest-posttest design. The efficacy of the web-integrated CinQASE learning model to train higher-order thinking skills is calculated with the eta-squared effect size for a paired sample t-test. In contrast, the description of students' achievement in higher-order thinking is viewed from the percentage of the average score of the student learning outcomes test on items that have C4 and C5 domains. The results showed that using the model had moderate effectiveness (η2 = 0.94) and helped train students’ ability to think at a higher level, as shown by an increase in students’ achievement in higher-level thinking in the good category. Thereby, the use of the web-integrated CinQASE learning model can supply considerable benefits in training students’ higher-order thinking skills. These results also write down a practical need for teachers and schools to implement learning approaches that use the potential of technology to enhance learners’ higher-order thinking skills in the 21st century.

Exploring the impact of online and offline teaching methods on the cognitive abilities of medical students: a comparative study

BackgroundOnline education has become increasingly popular, but research on the effectiveness of different teaching models in developing cognitive skills is limited. This study investigated the relationship between different teaching models (online and offline) and the development of cognitive skills in clinical medicine students.MethodsSurvey data were collected from 2018 entry students who participated in online teaching and 2019 entry students in offline teaching at Xiangya School of Medicine, Central South University. National Quality Open Courses (NQROC) were provided to both groups of students. The study examined the total score of physiology final exam, score of each type of question, and NQROC learning engagement in different score segments under the two teaching models. Non-parametric statistical methods were utilized to analyze the total score of physiology final exam, score of each type of question, and the NQROC learning engagement. Spearman’s rank correlation was utilized to analyze the relationship between the score of physiology final exam and the students’ NQROC learning engagement.ResultsThe study found no statistically significant difference in the total score, short-answer questions (SAQs) score, and case study questions (CSQs) score between online and offline teaching models. However, the multiple-choice questions (MCQs) score was higher in the online teaching model (Z=-4.249, P < 0.001), suggesting that online teaching may be an effective way to improve lower-order cognitive skills among students. In contrast, low-achieving students had higher total scores (Z=-3.223, P = 0.001) and scores in both MCQs (Z=-6.263, P < 0.001) and CSQs (Z=-6.877, P < 0.001) in the online teaching model. High-achieving students in the online teaching model had higher total scores (Z=-3.001, P = 0.003) and MCQs scores (Z=-5.706, P < 0.001) but lower scores in CSQs (Z=-2.775, P = 0.006). Furthermore, students’ NQROC learning engagement was greater in the online teaching model.ConclusionsThe results of this study suggested that online teaching was not statistically significantly different from offline in cognitive domains and was more desirable than offline in strengthening lower-order cognitive skills. However, it was important to note that offline teaching may be more effective in reinforcing higher-order cognitive skills among high-achieving students. In conclusion, this study provided important insights into the effectiveness of different teaching models in developing cognitive skills among medical students and highlighted the potential benefits of online teaching in enhancing students’ lower-order cognitive skills.

Analysis of Grade 10 Opportunity-to-Learn Dynamics of Market from Two Economics Textbooks

This theoretical paper analyses the Opportunity-to-Learn (OTL) Dynamics of Market offered in two Grade 10 economics textbooks. The study asked whether opportunities provided in economics textbooks improve learners’ performance in Dynamics of Market. Following a Content Analysis approach, three OTL variables; Content organization, Content coverage and Tasks were used as the theoretical framework to couch this study. One of the findings was that Less High-Order tasks were found in both textbooks hence do not develop learners’ Higher order Cognitive skills. The overall study showed that the analysed economics textbooks do not offer enough OTL Dynamics of Market. This study is significant because it helps both educators and policy makers to explore more on ‘must-have features’ of economics textbook before it could be prescribed for use. It makes contribution to international literature on textbook analysis in the field of economics education, this is rare in economics education studies. It is recommended that Higher order tasks should be included in future editions of the Grade 10 economics textbooks thus, suggesting the need to analyse economics textbooks in high schools for effective learning. Although the study analysed specific textbooks, its findings shed light on how learning opportunities relate to performance more generally.

Application of Scientific Based Modules to Improve High Order Cognitive Skills and Self Efficacy of Learners on Circulation System Material

Students who are at a higher level such as high school should already have higher knowledge or known as High Order Cognitive Skills (HOCS). Within the scope of education, self-efficacy is very important because it can affect cognitive processes, motivation, actions and student achievement. This study aims to determine the differences in high order cognitive skills and self-efficacy in the experimental and control classes. This research is quantitative through a Quasi-Experimental design with a Non-randomized control group, pretest-posttest design. Sampling used a purposive sampling technique with a total sample of 124 students who were divided into two groups, namely the control group and the experimental group. The instrument used to measure HOCS uses multiple choice questions of 40 test questions while self-efficacy is measured using a self-efficacy questionnaire. Data analysis used Ancova at a significance level of 0.05. The results showed that there were differences in the increase in high order cognitive skills and there were differences in students' self-efficacy on circulation system material in class XI after implementing scientific-based modules. The conclusion from this study is that scientific-based modules can be applied to improve students' high order cognitive skills and self-efficacy in circulation system material.

Use of Active Learning During Emergency Remote Teaching in COVID-19 Pandemic.

The mandatory isolation caused by COVID-19 required the adoption of emergency remote teaching, which caused difficulties for instructors, especially for those who use active learning that depends on student participation in class. This study aimed to investigate the ability of instructors to apply active learning effectively given the pandemic context. This was a cross-sectional observational study carried out in an undergraduate medical school. The sample was composed from one to three classes of 28 instructors that were observed synchronously. Each class was analyzed using a form created from an adaptation of the PORTAAL tool, aiming to evaluate quantitatively essential elements for active learning. We observed that the mean times devoted to activities and active participation of students were 54.8% and 33.1% of the total class time, respectively. Among the time spent in student interactions, the intra-group demanded the highest percentage of the class time. Additionally, 22.0% of the activities presented a high level in Bloom's taxonomy and there was a positive correlation between the percentage of activities at higher Bloom levels and the percentage of class time with student participation, intra-group or between-group interactions, supporting the use of higher-order cognitive skills in a collaborative and student-centered context. In conclusion, our findings indicate that some instructors were able to apply essential elements for an active and collaborative learning even during the emergency remote teaching.

Remote online open book examinations: through the lenses of faculty and students in health professions programmes

BackgroundOpen book examinations have been used to assess students’ higher order cognitive skills. These examinations can be conducted online remotely with the advancement of technology. However, there are concerns regarding its validity and reliability particularly if the examinations are not proctored. The objective of this study was to explore the perceptions of faculty and students in health professions programmes about remote online open book examinations (ROOBE).MethodsSemi-structured interviews were conducted among 22 faculty staff who were involved in ROOBE in health professions programmes. All interviews were audio recorded, transcribed verbatim and analysed using a thematic analysis approach. The perceptions of 249 medical students were obtained using an online questionnaire after they completed ROOBE.ResultsThe faculty agreed that open book examinations could promote students’ higher order cognitive skills and reduce students’ stress. However, they were concerned about students’ academic integrity during non-invigilated ROOBE which could affect recognition by accreditation and professional bodies. The shift from traditional practice of closed-book examinations to ROOBE required change management with the support of guidelines and faculty training. Majority of the students claimed that the examinations were challengingas they assessed their ability to apply knowledge in real world problems. Nevertheless, they preferred ROOBE due to less anxiety and memorisation, andmore emphasis on problem solving skills. The shortcomings were insufficient time for information searching duringexaminations and uncertainty in preparedness for future practice as theyfocused less on memorisation of factual knowledge during examinationpreparation. Cheating among peers andinternet instability during non-invigilated ROOBE were the concerns highlightedby some students.ConclusionsFaculty and students expressed favourable views about ROOBE in promotinghigher order cognitive skills. Adequatetechnological support was essential during ROOBE. While there was a need toaddress issues related to academic integrity, ROOBE could be included as anauthentic assessment within the systems of assessment.

Effectiveness of an analogy-containing video project to reinforce pharmacy students' learning of kidney physiology

Background and purposeKidney physiology is one of the most difficult topics covered in health professions education. This study examined the effectiveness of an analogy-containing video project to reinforce pharmacy students' knowledge of kidney physiology. Educational activity and settingStudents were assigned in groups to create a video that used analogy to explain kidney physiology processes. Survey responses, a rubric, and an objective test were used to assess the project's effectiveness. FindingsStudents generated various analogies, such as making tea or coffee, cars and roads, and the college application process to explain kidney physiology. Most of the submitted videos successfully met all criteria in the rubric. All students believed that the project was effective, to varying degrees, in reinforcing their knowledge. Weak to moderate positive correlations were found between time spent on the project and perceived effectiveness of use of analogy to accomplish most of the learning objectives investigated. Students reported that the project engaged both lower- and higher-order cognitive skills. Quiz score analysis showed higher average scores of students on physiology-related questions compared to non-physiology items. Notably, this improvement was most marked for bottom-performing students. About 60% of students recommended the use of videos as a class assignment, but only 50% believed that they should be used as an alternative to quizzes. SummaryAn analogy-containing video project effectively reinforced student learning of kidney physiology. Further studies are needed to verify the effectiveness of this technique compared to other didactic and learning approaches.