Assessment Of Critical Thinking Research Articles

Towards a framework of critical thinking for assessing EAP speaking

As one of the fundamental skills of the 21st century, critical thinking (CT) is a topic of considerable interest within the domain of assessing English for academic purposes (EAP). Recent literature on EAP instruction and assessment indicates that EAP has evolved beyond a strict focus on language improvement to encompass discourse competence and broader academic literacy development, with CT playing an important role in academic communication. A fundamental consideration in educational assessment is the construct that defines the knowledge, skills, or abilities to be assessed. However, CT has not received due attention in the research literature on EAP speaking assessment, thus raising a legitimate concern about the underrepresentation of the academic speaking construct. This study draws on Macqueen's (2022) distinction between theoretical, stated, perceived, and operationalized assessment constructs, with an aim to describe the EAP speaking assessment construct by including the concept of CT. By establishing alignment between what the literature indicates (theoretical), what assessment requires (stated), and how people understand assessment (perceived), a CT assessment framework is proposed to reflect how test takers experience it (operationalized). Our study contributes to a more nuanced conceptualization of CT in the context of EAP speaking, with implications for EAP speaking test development.

Teaching critical thinking to Chinese students in English as a foreign language writing class: A review

<span lang="EN-US">Critical thinking (CT) is important for English as a foreign language (EFL) writing. However, the EFL writings of Chinese students are usually negatively commented on in terms of CT in writing. The study reviewed relevant studies on how CT in writing was taught to Chinese EFL learners at the tertiary education level. It shows most EFL writing teachers adopted existing western CT definitions in a general knowledge background and mainly used teacher-centered assessment techniques to teach CT. Students’ CT development was mostly assessed by self-designed CT assessments. Facing the globalization of CT teaching, the study thus updates the assessment techniques to meet the increasing needs of CT teaching in EFL writing in China. Relevant suggestions are also given to education practitioners.</span>

Adapting or adopting? Critical thinking education in the East Asian cultural sphere: A systematic integrative review

Confucian culture influences the teaching effect of critical thinking (CT). Although education systems have taken measures to revise curricula and instruction, the lack of CT among students in some countries belonging to the East Asian cultural sphere is widely criticized, and it is generally believed that their education systems fail to develop CT skills in their students. Given these challenges, this study aims to analyze how the concept, assessment and instruction of CT has been included in education in these countries using an integrative systematic review. Based on a series of inclusion criteria, 62 articles were included, and their quantitative and qualitative characteristics were analyzed. Although cultural influence has been emphasized, researchers and educators generally adopt Western definitions of the concepts of CT without questioning their transferability. With regard to assessment, researchers rarely develop local instruments, and it is more common to adopt authoritative tools developed in Western countries without adaptation and without examining their reliability and validity. With regard to instruction, researchers rarely adapt teaching strategies based on local conditions, although culture and language affect teaching efficacy. The review provides a critical analysis of the contextualization of CT in the literature and outlines new directions for interested researchers and educators in the field of CT teaching.

Building consensus on critical thinking assessment tool for undergraduate nursing students in a socialist low-middle income country: A delphi study

AimWe aimed to build consensus on a holistic critical thinking (CT) assessment tool involving cognitive skills and disposition for undergraduate nursing students in a socialist low-middle income country, Lao People’s Democratic Republic (Lao PDR). BackgroundUniversity degree nursing education program in Lao PDR has been started since 2011 to strengthen the ability to think as professional nurse, and nursing regulations have progressively developed last decade. However, there are still few studies assessing the CT ability of nursing students in Lao PDR. DesignDelphi study. MethodsWe conducted three rounds of Delphi survey of 14 panel members consisting of nursing faculty members and clinical supervisors. The first and second survey rounds involved a combination of a series of questionnaires and face-to-face interviews to obtain ideas or opinions about descriptions of the CT assessment tool. The third round was the meeting with 13 panel members to reach a consensus on the descriptions of the CT assessment tool. ResultsThe number of descriptions changed from 29 to 30 under seven assessment criteria of the cognitive skills and from 34 to 32 under 10 assessment criteria of the habits of mind as disposition. The panel provided active comments on the descriptions in terms of social, cultural, and educational backgrounds. Especially the descriptions of criteria applying standards and creativity were discussed from practical viewpoints such as the hierarchical social structure in hospital, limited educational resources, and the cultural value of harmony. ConclusionsThe holistic CT assessment tool that resulted from the consensus is anticipated to help Laotian nursing educators to assess undergraduate nursing students’ CT ability more efficiently with confidence at both the university and hospitals continuously. Subsequent implementation of the CT assessment tool in a socialist low-middle income country is warranted.

A teachers’ based approach to assessing the perception of critical thinking in Education university students based on their age and gender

In the 21st century, critical thinking (CT) is regularly presented as one of the most important competences to be developed by a majority of educational institutions. Teachers are expected to change and enrich their teaching and learning methodologies so that students could face future challenges. Nonetheless, few are the instruments that measure the perception of critical thinking based on teachers’ conception. The aim of this study is to design and validate an instrument for the assessment of CT in university students based on the conception of CT that university teachers have. For this study, a total of 312 Spanish university students have participated. Based on a good model fit from a Confirmatory Factor Analysis and good reliability indices, the results provide strength to the theoretical model to evaluate critical thinking in university students formed by six dimensions (Analyzing/Organizing; Reasoning/Argumenting; Questioning/Asking oneself; Evaluating; Positioning /Taking decisions, and Acting /Committing oneself) and 42 items. Similarly, age was not a predictor variable for the different dimensions; while gender was statistically in favor of women in some of the dimensions, and tendentially, the dimension of Positioning/Taking decisions, in favor of men. However, despite these differences, the model guaranteed its factorial invariance. These findings have important pedagogical implications for universities in particular, and educational institutions in general, when developing curricula and teaching plans that focus on the development of students’ critical thinking.

Investigating EFL writers’ critical thinking performance across languages

Understanding how writing in different languages affects one's critical thinking (CT) performance is of high importance for both the teaching and assessment of CT. Previous research has identified a close connection between CT and writing as well as the effectiveness of evaluating CT through the proxy of writing (e.g., Bean, 2011; Dong, 2017, 2018; Rahmat, 2020); however, little is known about whether and how one's CT performance varies when writing in different languages. This mixed-methods study was designed to fill this gap by comparing 31 Chinese English-as-a-Foreign-Language (EFL) students’ CT scores, determined through evaluation of their essays written in English and Chinese. The results showed that the participants demonstrated significantly better CT abilities when they wrote in their L1 (t = 8.105, p < 0.01); however, there was no significant correlation between students’ L1 CT and L2 CT scores. In addition, the study identified a number of factors that significantly influenced students’ decision-making in their L1 and L2 writing processes and affected their CT performance in aspects such as clarity, precision, breadth, depth, and relevance. This study contributes to a dynamic conceptualization of EFL writers’ CT performance across languages and highlights the need to recognize the differences between multilingual students’ CT abilities and CT performance. The study also provides practical implications for the teaching and assessment of CT in multilingual contexts.

Computer-based performance approach for critical thinking assessment in children.

Critical thinking (CT) has been identified as one of the key skills in the modern world. Despite a variety of CT assessment systems for secondary school and higher education, there is a shortage of CT assessments for children. To fill the gap, we developed a computer-based performance assessment (CPBA) of CT with automatic scoring. In the study, we analysed: (1) To what extent is the internal structure of the critical thinking assessment tool confirmed? (2) What are the patterns of students' behaviour within the CPBA? (3) To what extent is the convergent validity of the CPBA supported? The sample consists of 1689 students, aged 9-11 years. To answer the research questions, the following quantitative data analysis methods were selected. The Bayesian Network was applied to investigate the internal structure. Latent Class Analysis was used to investigate the patterns of students' behaviour. To verify the convergent validity, the students were asked to solve the standardized math and language literacy tests. The theoretically expected internal structure of the CPBA was confirmed: the two-factor structure (the Analysis of information and the Making inferences subskills of CT) describes the data appropriately. Moreover, CT was described not as a list but as a sequence of cognitive operations. Four different strategies that lead to different results of CT were revealed. A weak positive correlation between the results of the CPBA and the achievement tests confirmed the test validity.

Critical thinking (CT) in sustainable higher education: Ensuring consistent CT perception-practice and identifying gaps between college instructors' and students' perceptions in advanced academic writing courses in the UAE

Sustainable higher education is expected to assist in achieving sustainable development goals (SDGs) by developing several academic competencies, including CT skills. However, sustaining the development of college students' CT skills requires ensuring a consistent relationship between CT perceptions and practices. Hence, this study seeks to examine college instructors' actual CT practices in light of their stated perceptions. Furthermore, it compares college instructors' and their students' CT perceptions, practices, and assessment in advanced academic writing courses in the UAE. A mixed-method design method was used. Six college English writing instructors and their students (n = 164) responded to the teacher and student questionnaires, five out of the six were observed, and the same six were interviewed at the end. Overall, instructors' and students' perceptions and practices were similar except for a few areas, especially about CT explicit instruction and modeling. Furthermore, an inconsistent relationship was identified between instructors' CT explicit instruction perceptions and their actual practices. While instructors supported explicit CT instruction in theory, five out of six confirmed they had never defined or modeled CT in practice. Finally, considering the significant role of CT in achieving sustainable development goals and students' calls for more CT framing, the study provided several recommendations for higher education CEOs, academic leaders, and college instructors.

Reliability of the Critical Thinking in Communication Science and Disorders: A Content-Specific Critical Thinking Assessment

Introduction: Students and professionals in communication sciences and disorders (CSD) need to exhibit good critical thinking (CT) skills when engaged in clinical tasks. CSD clinicians must make decisions that are free from biases and support their claim with facts. Thus, CSD clinicians need to be trained to question their clinical practices and to skeptically evaluate new practices that develop. A content-specific CT test can help determine if students are developing these skills. However, to date, no such content-specific CT assessment exists for CSD. The purpose of this study was to determine the reliability of the current version of a specific content CT assessment, the Critical Thinking in Communication Sciences and Disorders (CTCSD). Methods: A sample of 150 CSD graduate students enrolled in three programs participated. They completed an online Qualtrics survey that consisted of the CTCSD. They completed the Qualtrics survey twice, once at the beginning of a semester and once at the end. The participant responses were independently scored by two research associates. The data were analyzed for reliability in three ways. Intra-subject reliability was assessed by comparing scores across the two testing sessions. Internal consistency of the items to measure a common construct was assessed using Cronbach’s alpha and Guttman’s Lambda 6. Inter-rater reliability was assessed using Cohen’s Kappa coefficient. In addition, the time used to complete the survey was analyzed. Results: The students from the three programs scored similarly on the CTCSD. High reliability ratings occurred for the intra-subject, internal consistency, and inter-rater measures. Discussion/Conclusion: The results indicate the reliability of the CTCSD. In combination with previous results indicating the face, construct, and criterion validity of the CTCSD, it appears to have psychometric strength. The CTCSD may help academic and clinical faculty select learning activities and focus feedback to their graduate students in order to reinforce skills the students exhibit and to develop other skills.

Assessing students’ critical thinking skills viewed from cognitive style: Study on implementation of problem-based e-learning model in mathematics courses

The digitalization system that continues to roll has brought changes to the learning system, where face-to-face learning is replaced by an online system. On the one hand, learning experiences to acquire critical thinking (CT) skills as one of the essential skills of the 21st century must also be encouraged. The objective of this study is to assess students’ CT skills in terms of cognitive style by implementing the problem-based e-learning (e-PBL) model in mathematics courses. This study is an evaluative study with an experimental approach, where as many as 28 students as research samples were taken purposively from Mandalika University of Education, Indonesia. A set of instruments was prepared to measure every aspect of CT and cognitive style, including descriptive and statistical data analysis so that the results of the CT assessment were found. In general, the results of the CT evaluation show that e-PBL is effective in improving students’ CT skills, so this is a recommendation to use e-PBL widely and intensively.

Developing an e-learning-based critical-thinking assessment as a physics learning evaluation media with Kahoot! interactive quiz

This study aimed to determine process, feasibility, validity, and reliability of the developed E-learning-based critical thinking assessment as a physics learning evaluation media with Kahoot!. The Research and Development method used was by Borg & Gall. The instrument used was a questionnaire. The type of data produced was quantitative data. The final product was declared feasible with the obtained score percentage of 85.89% by media experts, 80.50% by the material expert, and 94.93% by information and technology expert. The developed product was deemed attractive with the obtained attractiveness score percentages in the small-group trial of 86.10% and the field trial of 83.03%. The items of the product were reliable. The objective of this study was only to develop a physics learning evaluation product using Kahoot! and did not seek the influence of the use of the developed product. Therefore, for further research it can be found the effect of using the following products on student achievement. Based on the research results, the developed product in the form of an E-learning-based critical thinking assessment as a physics learning evaluation media with Kahoot! interactive quiz is said to be feasible and can be used to evaluate physics learning activities.

Relevance of the socio-cultural perspective in the discussion about critical thinking

Critical thinking (CT) is considered a key skill for success in the 21st century. Worldwide educational policies advocate the promotion of CT, and researchers across different fields have been involved in a wide debate on its definition, without reaching an agreement. Currently, research has not adequately addressed CT assessment, nor the way in which it should be taught. In the present work, an overview of the topic is provided, as well as an evaluation of the practices, in order to provide researchers or practitioners (particularly those involved in primary school education) a reference for the development of further theories and methods about CT in education. We consider CT from the perspective of philosophy, cognitive psychology, and education sciences, and in addition, we propose the inclusion of a fourth, which could be referred as socio-cultural pedagogic perspective, due to its important implications in teaching and assessment practices.

Assessing Critical Thinking in Open-ended Answers: An Automatic Approach

The role of Higher Education (HE) is growingly acknowledged for the promotion of Critical Thinking (CT). Constructed-response tasks (CRT) are recognized to be necessary for the CT assessment, though they present problems related to scoring quality and cost (Ku, 2009). Researchers (Liu, Frankel, Roohr, 2014) have proposed using automated scoring to address the above concerns. The present work is aimed at comparing the features of different Natural Language Processing (NLP) techniques adopted to improve the reliability of a prototype designed to automatically assess six sub-skills of CT in CRT: use of language, argumentation, relevance, importance, critical evaluation and novelty (Poce, 2017). We will present the first (1.0) and the second (2.0) version of the CT prototype and their respective reliability results. Our research question is the following: Which level of reliability are shown respectively by the 1.0 and 2.0 automatic CT assessment prototype compared to expert human evaluation? Data collection is realized in two moments, to measure respectively the CT prototype 1.0 and 2.0 reliability from a total of 264 participants and 592 open-ended answers. Two human assessors rated all of these responses on each of the subskills on a scale of 1-5. Similarly, NLP approaches are adopted to compute a feature on each dimension. Quadratic Weighted Kappa and Pearson product-moment correlation were used to evaluate the between-human agreement and human-NLP agreement. Preliminary findings based on the first data set suggest adequate level of between-human rating agreement and a lower level human-NLP agreement (r .43 for the subscales of Relevance and Importance). We are continuing the analysis of the data collected in the 2nd step and expect to complete them in June 2020.

Developing a rubric to assess critical thinking in a multidisciplinary context in higher education

Critical thinking (CT) is a generic attribute that is greatly valued across academic disciplines in higher education, and around the globe. It is also defined as one of the graduate attributes of higher education for the sample private university where this research was conducted, as it is perceived that CT helps the graduate to become ‘engaged citizens’ in the twenty-first century. Despite the well-documented importance of CT, its assessment remains a challenge. This study addresses this challenge through the systematic development and field-testing of a rubric for assessing critical thinking in a multidisciplinary context in higher education. A multidisciplinary group of faculty (i.e. education, nursing, medicine) from the sample university partnered with a policy research group in Canada to translate this plan into action. The development of the assessment tool followed a multi-step process including: (i) identification of the main elements of CT; (ii) choice of a rubric format; (iii) adaptation of the currently available relevant rubrics; and, (iv) field testing and establishment of the reliability of the rubric. The process resulted in the development of a holistic template, the Assessment of Critical Thinking (ACT) rubric. Two versions of the rubric have been field tested on a sample (n=59) of students drawn from different entities of the sample university. The data collected was subjected to psychometric analysis which yielded satisfactory results. This was a modest attempt to develop an assessment tool to guide multidisciplinary faculty members in teaching and assessing CT by assisting them to make decisions about the level of their students’ CT skills through a combination of numerical scores and qualitative description. It may also empower them to make self-initiated, conscious efforts to improve their classroom practice with reference to CT. The ACT rubric provides an anchoring point to start working on the daunting yet doable task of developing and fine-tuning both the assessment measures of CT and interventions to promote CT based on the assessment findings. Future research may not only provide robust evidence of the reliability and validity of the ACT rubric for a larger and varied sample but also help in making informed decisions to enhance teaching and learning of CT across entities of the sample university.&#x0D; &#x0D; How to cite this article: &#x0D; BHUTTA, Sadia Muzaffar; CHAUHAN, Sahreen; ALI, Syeda Kausar; GUL, Raisa; CASSUM, Shanaz; KHAMIS, Tashmin. Developing a rubric to assess critical thinking in a multidisciplinary context in higher education. Scholarship of Teaching and Learning in the South. v. 3, n. 1, p. 6-25, Apr. 2019. Available at: https://sotl-south-journal.net/?journal=sotls&amp;page=article&amp;op=view&amp;path%5B%5D=69&amp;path%5B%5D=34&#x0D; &#x0D; This work is licensed under the Creative Commons Attribution 4.0 International License.&#x0D; To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

A Convergent Class Model Development and a Study of Evaluation Method for Critical Thinking Formulation: Through a Consideration of Hardy’s Tess and Foucault

This study explores the effect of a humanities-based convergent class model (HBCCM) on critical thinking (CT) when Kakao-Talk is utilized as a virtual learning space to acquire new knowledge using flipped learning (FL). Its objective is to promote EFL undergraduates’ (n=51) participation in face-to-face discussion activities by getting them to prepare before coming to class. The preliminary learning contents and conceptual knowledge organized into specific argumentative topics based on Thomas Hardy’s Tess of the D’Urbervilles and Foucault’s philosophical perspective of Sexuality, Madness and Power were provided via Kakao-Talk. Students could experience a self-regulatory learning process outside of class and develop their ideas with peers' through in-class discussion. While the collaborative learning through group activities promoted social interaction, interdependence and critical analysis, the students self-evaluated their ideas clearly. Qualitative and quantitative methodologies including Watson-Glaser's (W-G) CT assessment were conducted to evaluate the change of CT skills. The results showed the significant interaction of the CT skills. In particular, the students improved the skills of recognizing assumption, deductive reasoning and evaluating arguments as they experienced CT-efficacy through this learning process. Accordingly, HBCCM may contribute to the higher education for activating CT and the improvement of a teaching and learning method.

Self-Assessment of Critical Thinking Skills in EFL Writing Courses at the University Level: Reconsideration of the Critical Thinking Construct

The Critical Thinking (CT) component has by now secured a key place within EFL curriculum aims and objectives. The integration of a CT dimension into the teaching of the writing skill in particular has received considerable attention in research. However, research has also pointed to the failure of assessment practices to evaluate CT development. It is within this context that the present work advocates a standard based approach to the assessment of CT in EFL writing that aligns assessment criteria to the critical abilities articulated in the learning outcomes (i.e., standards). This fosters a conception of CT measures that associates components of CT with higher order writing skills. Accordingly, the first objective of the study is to empirically test the relationship between CT dispositions and metacognitive strategy use in an attempt to establish a model of writing (self-)assessment that combines these two dimensions. As assessment is viewed as a formative evaluation process subservient of learning, the study also targets the students’ self-assessment strategies during the writing process. To this purpose, a questionnaire has been designed, and administered to 100 students at the Faculty of Letters and Human Sciences in Rabat to tap their perceptions and use of CT skills. The data analysis revealed that critical thinking development and assessment are metacognitive in nature; it follows that metacognitive skills such as planning, self-evaluating and reflecting are to be used as an essential vehicle in the development of Critical Thinking skills. This points to the paramount role of CT-informed formative (self-)assessment practices in benefiting ELT writing learners.

Measuring Critical Thinking in Physics: Development and Validation of a Critical Thinking Test in Electricity and Magnetism

Although the development of critical thinking (CT) is a major goal of science education, adequate emphasis has not been given to the measurement of CT skills in specific science domains such as physics. Recognizing that adequately assessing CT implies the assessment of both domain-specific and domain-general CT skills, this study reports on the development and validation of a test designed to measure students’ acquisition of CT skills in electricity and magnetism (CTEM). The CTEM items were designed to mirror the structural components of items identified in an existing standardized domain-general CT test, and targeted content from an introductory Electricity and Magnetism (E&M) course. A preliminary version of the CTEM test was initially piloted on three groups of samples: interviews with physics experts (N = 3), student cognitive interviews (N = 6), and small-scale paper and pencil administration (N = 19). Modifications were made afterwards and the test was administered to a different group of second-year students whose major was mechanical engineering (N = 45). The results showed that the internal consistency (Cronbach’s α = .72) and inter-rater reliability (Cohen’s kappa = .83) of the CTEM test are acceptable. The findings overall suggest that the CTEM test can be used to measure the acquisition of domain-specific CT skills in E&M, and a good basis for future empirical research that focuses on the integration of CT skills within specific subject matter instruction. A broader CT assessment framework is proposed and possible research questions that can be addressed through the CTEM test are discussed.

The Relationship between Critical Thinking and Language Learning Strategies of EFL Learners

Critical thinking (CT) is being increasingly discussed in language learning and teaching. Discussions around CT are particularly influencing English for Academic Purposes (EAP) which the teachers’ aim for the students is to help them learn all of the skills necessary to succeed in the university career. Accordingly, the study mainly surveys the effect of receiving critical thinking trainings on degree students through the English for Academic Purposes (EAP) program. Participants of the study included 256 freshmen students from four departments at a leading university in China. The experiment group studied EAP, while the control group studied General English within one academic semester. In addition to CT assessment, the two groups were also compared in regards with Language Learning Strategy (LLS). The significant contribution of the study might be to reveal the possible relationship between CT and LLS. The results and discussions would be conducive for language practitioners and administrators.

Critical Analysis

This study compared changes in psychology and philosophy classes in two distinct components of critical thinking (CT): general skills and personal beliefs. Participants were 128 undergraduates enrolled in CT in psychology, other psychology courses, or philosophy courses. CT and philosophy students significantly reduced beliefs in paranormal phenomena at the end of the semester compared to other psychology students. Only philosophy students improved on the Watson–Glaser Critical Thinking Appraisal. The Watson–Glaser may not fully measure CT emphasized in psychology, but psychology instructors can still effectively teach students to examine their own beliefs and think differently in their daily lives. Differentiated assessment of CT is important as instructors evaluate student learning against specific goals.

Validation of a Scenario-Based Assessment of Critical Thinking Using an Externally Validated Tool

With medical education transitioning from knowledge-based curricula to competency-based curricula, critical thinking skills have emerged as a major competency. While there are validated external instruments for assessing critical thinking, many educators have created their own custom assessments of critical thinking. However, the face validity of these assessments has not been challenged. The purpose of this study was to compare results from a custom assessment of critical thinking with the results from a validated external instrument of critical thinking. Students from the College of Veterinary Medicine at Western University of Health Sciences were administered a custom assessment of critical thinking (ACT) examination and the externally validated instrument, California Critical Thinking Skills Test (CCTST), in the spring of 2011. Total scores and sub-scores from each exam were analyzed for significant correlations using Pearson correlation coefficients. Significant correlations between ACT Blooms 2 and deductive reasoning and total ACT score and deductive reasoning were demonstrated with correlation coefficients of 0.24 and 0.22, respectively. No other statistically significant correlations were found. The lack of significant correlation between the two examinations illustrates the need in medical education to externally validate internal custom assessments. Ultimately, the development and validation of custom assessments of non-knowledge-based competencies will produce higher quality medical professionals.