Lower-order Questions Research Articles

Solving Not Answering. Validation of Guidance for Writing Higher-Order Multiple-Choice Questions in Medical Science Education

AbstractProblem-solving and higher-order learning are goals of higher education. It has been repeatedly suggested that multiple-choice questions (MCQs) can be used to test higher-order learning, although objective empirical evidence is lacking and MCQs are often criticised for assessing only lower-order, factual, or ‘rote’ learning. These challenges are compounded by a lack of agreement on what constitutes higher order learning: it is normally defined subjectively using heavily criticised frameworks such as such as Bloom’s taxonomy. There is also a lack of agreement on how to write MCQs which assess higher order learning. Here we tested guidance for the creation of MCQs to assess higher-order learning, by evaluating the performance of students who were subject matter novices, vs experts. We found that questions written using the guidance were much harder to answer when students had no prior subject knowledge, whereas lower-order questions could be answered by simply searching online. These findings suggest that questions written using the guidance do indeed test higher-order learning, and such MCQs may be a valid alternative to other written assessment formats designed to test higher-order learning, such as essays, where reliability and cheating are a major concern.

Is ChatGPT ‘ready’ to be a learning tool for medical undergraduates and will it perform equally in different subjects? Comparative study of ChatGPT performance in tutorial and case-based learning questions in physiology and biochemistry

Purpose Generative AI will become an integral part of education in future. The potential of this technology in different disciplines should be identified to promote effective adoption. This study evaluated the performance of ChatGPT in tutorial and case-based learning questions in physiology and biochemistry for medical undergraduates. Our study mainly focused on the performance of GPT-3.5 version while a subgroup was comparatively assessed on GPT-3.5 and GPT-4 performances. Materials and methods Answers were generated in GPT-3.5 for 44 modified essay questions (MEQs) in physiology and 43 MEQs in biochemistry. Each answer was graded by two independent examiners. Subsequently, a subset of 15 questions from each subject were selected to represent different score categories of the GPT-3.5 answers; responses were generated in GPT-4, and graded. Results The mean score for physiology answers was 74.7 (SD 25.96). GPT-3.5 demonstrated a statistically significant (p = .009) superior performance in lower-order questions of Bloom’s taxonomy in comparison to higher-order questions. Deficiencies in the application of physiological principles in clinical context were noted as a drawback. Scores in biochemistry were relatively lower with a mean score of 59.3 (SD 26.9) for GPT-3.5. There was no statistically significant difference in the scores for higher and lower-order questions of Bloom’s taxonomy. The deficiencies highlighted were lack of in-depth explanations and precision. The subset of questions where the GPT-4 and GPT-3.5 were compared demonstrated a better overall performance in GPT-4 responses in both subjects. This difference between the GPT-3.5 and GPT-4 performance was statistically significant in biochemistry but not in physiology. Conclusions The differences in performance across the two versions, GPT-3.5 and GPT-4 across the disciplines are noteworthy. Educators and students should understand the strengths and limitations of this technology in different fields to effectively integrate this technology into teaching and learning.

Performance evaluation of ChatGPT, GPT-4, and Bard on the official board examination of the Japan Radiology Society.

Herein, we assessed the accuracy of large language models (LLMs) in generating responses to questions in clinical radiology practice. We compared the performance of ChatGPT, GPT-4, and Google Bard using questions from the Japan Radiology Board Examination (JRBE). In total, 103 questions from the JRBE 2022 were used with permission from the Japan Radiological Society. These questions were categorized by pattern, required level of thinking, and topic. McNemar's test was used to compare the proportion of correct responses between the LLMs. Fisher's exact test was used to assess the performance of GPT-4 for each topic category. ChatGPT, GPT-4, and Google Bard correctly answered 40.8% (42 of 103), 65.0% (67 of 103), and 38.8% (40 of 103) of the questions, respectively. GPT-4 significantly outperformed ChatGPT by 24.2% (p < 0.001) and Google Bard by 26.2% (p < 0.001). In the categorical analysis by level of thinking, GPT-4 correctly answered 79.7% of the lower-order questions, which was significantly higher than ChatGPT or Google Bard (p < 0.001). The categorical analysis by question pattern revealed GPT-4's superiority over ChatGPT (67.4% vs. 46.5%, p = 0.004) and Google Bard (39.5%, p < 0.001) in the single-answer questions. The categorical analysis by topic revealed that GPT-4 outperformed ChatGPT (40%, p = 0.013) and Google Bard (26.7%, p = 0.004). No significant differences were observed between the LLMs in the categories not mentioned above. The performance of GPT-4 was significantly better in nuclear medicine (93.3%) than in diagnostic radiology (55.8%; p < 0.001). GPT-4 also performed better on lower-order questions than on higher-order questions (79.7% vs. 45.5%, p < 0.001). ChatGPTplus based on GPT-4 scored 65% when answering Japanese questions from the JRBE, outperforming ChatGPT and Google Bard. This highlights the potential of using LLMs to address advanced clinical questions in the field of radiology in Japan.

Performance of ChatGPT on a Radiology Board-style Examination: Insights into Current Strengths and Limitations.

Background ChatGPT is a powerful artificial intelligence large language model with great potential as a tool in medical practice and education, but its performance in radiology remains unclear. Purpose To assess the performance of ChatGPT on radiology board-style examination questions without images and to explore its strengths and limitations. Materials and Methods In this exploratory prospective study performed from February 25 to March 3, 2023, 150 multiple-choice questions designed to match the style, content, and difficulty of the Canadian Royal College and American Board of Radiology examinations were grouped by question type (lower-order [recall, understanding] and higher-order [apply, analyze, synthesize] thinking) and topic (physics, clinical). The higher-order thinking questions were further subclassified by type (description of imaging findings, clinical management, application of concepts, calculation and classification, disease associations). ChatGPT performance was evaluated overall, by question type, and by topic. Confidence of language in responses was assessed. Univariable analysis was performed. Results ChatGPT answered 69% of questions correctly (104 of 150). The model performed better on questions requiring lower-order thinking (84%, 51 of 61) than on those requiring higher-order thinking (60%, 53 of 89) (P = .002). When compared with lower-order questions, the model performed worse on questions involving description of imaging findings (61%, 28 of 46; P = .04), calculation and classification (25%, two of eight; P = .01), and application of concepts (30%, three of 10; P = .01). ChatGPT performed as well on higher-order clinical management questions (89%, 16 of 18) as on lower-order questions (P = .88). It performed worse on physics questions (40%, six of 15) than on clinical questions (73%, 98 of 135) (P = .02). ChatGPT used confident language consistently, even when incorrect (100%, 46 of 46). Conclusion Despite no radiology-specific pretraining, ChatGPT nearly passed a radiology board-style examination without images; it performed well on lower-order thinking questions and clinical management questions but struggled with higher-order thinking questions involving description of imaging findings, calculation and classification, and application of concepts. © RSNA, 2023 See also the editorial by Lourenco et al and the article by Bhayana et al in this issue.

Status of Enhancing Learners’ Higher Order Thinking Skills in the Teaching of Reading Skills at Ethiopian Higher Education: Analyzing Reading Exercises and Learners’ Perceptions

Abstract In the twenty-first century, the broad availability of information and technological innovation has resulted in a paradigm shift in the goal of education. As a result, in order to produce engaged citizens capable of facing the demands and obstacles of the century, boosting learners' higher cognitive skills has become a priority in modern education. Language teachers' responsibilities in modern language teaching have expanded to include improving students' thinking skills. In line with this idea, this study was aimed at examining the role teaching reading skills at a higher education institution has in enhancing learners’ higher-order thinking skills. In order to achieve the main objective of the study, an analysis was carried out on the levels of thinking that questions in reading exercises aim to elicit from the learners and how the students perceive the reading exercises from the perspective of enhancing higher-order thinking skills. The findings of the study show that the reading exercises are dominated by lower-order questions; there are fewer higher-order questions in the exercises. Thus, the reading exercises are not capable of enhancing learners’ higher-order thinking skills. The students also have the perception that the reading exercises are not meant to develop their higher-order thinking skills. There is no statistically significant difference between male and female participant students among their perceptions identified in the study.

The effect of curricular reform on gross anatomy laboratory examination performance: An institutional analysis.

Many medical schools have undergone curricular reform recently. With these reforms, time spent teaching anatomy has been reduced, and there has been a general shift to a pass/fail grading system. At Indiana University School of Medicine (IUSM), a new curriculum was implemented in fall 2016. The year-long human gross anatomy course taught in 2015 was condensed into an integrated, semester-long course starting in 2016. Additionally, the grading scale shifted to pass/fail. This study examined first-year medical student performance on anatomy practical laboratory examinations-specifically, among lower-order (pure identification) questions and higher-order (function, innervation) questions. Participants included medical students from a pre-curricular reform cohort (year 2015, 34 students) and two post-curricular reform cohorts (years 2016, 30 students and 2017, 33 students). A Kruskal-Wallis ANOVA test was used to determine differences of these questions among the three cohorts. Additionally, 40 of the same lower-order questions that were asked on gross anatomy laboratory examinations from medical student cohort year 2015 and year 2016 were further analyzed using an independent samples t-test. Results demonstrated that the pre-curricular reform cohort scored significantly higher on both lower-order (median=81, p < 0.001) and higher-order questions (median=82.5, p < 0.05) than both post-curricular reform cohorts. Additionally, when reviewing the selected 40 similar questions, it was found that the pre-curricular reform cohort averaged significantly higher (82.1 ± 16.1) than the post-curricular reform cohort from 2016 (69.3 ± 21.8, p=0.004). This study provides evidence about the impact of curricular reform on medical student anatomical knowledge.

Intra-observer evaluation of teacher-student questioning behaviours: A case study of the quality and frequency of questions administered in physical education theory class

Questioning in teaching and learning environment is an effective way of encouraging learners to contribute to the subject matter. A lesson is incomplete without questioning learners or offering learners the opportunity to question the teacher. Teachers should engage learners in high-order questions that trigger critical thinking in teaching and learning environment. However, studies that focused on judging the quality of questions generated in classroom setting revealed that most classroom questions fell within lower-order of thinking (Caspari-Sadeghi et al., 2021; Momsen et al., 2010). Again, earlier study conducted by a group of psychologists revealed that over ninety-five percent (95%) of the questions learners encountered required them to think at the lowest possible level - the recall of information (Bloom et al., 1956). Based on Bloom’s original work relating to cognitive levels of thinking, we conducted a descriptive analysis of the quality and frequency of questions administered in a lesson involving 38 students of a College of Education and one physical education teacher (intern of a University in Ghana). Specifically, we concentrated on classroom behaviours that related to the quality and frequency of questions asked in physical education (PE) classroom setting. From a video-recorded lesson, data on questioning behaviours of students and the teacher in a theory physical education class were collected via self-developed event recording instrument. An expert in research (senior university faculty) inspected and approved the instrument for collecting accurate data for the study. Data were descriptively analyzed and presented in tables and figures. Analysis of intra-observer results revealed that most of the classroom questions were lower-order questions that did not promote critical thinking among students. In order of magnitude (lowest level to highest level), questions asked by the teacher occurred at the level of knowledge, understanding, synthesis and evaluation. Findings also revealed that there was no question asked at the level of application and analysis by the teacher in the entire 35 minutes lesson. The study showed that students did not ask any question(s) in the entire duration of the lesson. We recommend that questions of various levels should form part of planning decisions of every teacher.

A clinical supervision intervention in questioning: an action research study

ABSTRACT This article presents the outcomes of a project, which aimed to develop the use of questioning in one teacher’s. practice using a professional development intervention termed ‘clinical supervision’. As well as being a project which adopted an action research approach, this use of clinical supervision is, we argue, a form of professional collaborative support between more- and less-experienced colleagues, akin to the kinds of relationships typical of collaborative action research. Questioning, as a common teaching technique second only to lecturing, is evidenced to be a powerful teaching strategy when used competently. Yet, many teachers struggle to do so by spontaneously asking unplanned questions during a lesson, over-using lower-order questions or even furnishing the answers themselves before waiting for student responses. The current study thus aimed to examine the routine pre-intervention questioning strategies used by the participating teacher and then document the outcomes of the clinical supervision intervention that targeted the development of her questioning skills. Findings indicated that the teacher’s pre-intervention use of questioning was typical of the kinds of errors that some teachers make as evidenced in existing literature; while post-intervention findings revealed notable improvement in the teacher's practice of the targeted questioning skills.

Evaluation of student-teacher questions: A study of pedagogical practices relating to proportions and rate per-minute of questions asked in physical education class setting

Questioning is one of the pedagogical strategies that ensures that students appropriately engage in relevant activities in teaching and learning environment. Questioning plays a critical role in refreshing memories of learners on their previously acquired knowledge or in the acquisition of new knowledge while a lesson is in progress. Regular questioning of students is a good practice to help learners think for themselves instead of being “spoon-fed” by the teacher. However, over concentration on lower-order questions has a dire consequence on the thinking skills of students. In this study, we used a descriptive approach to examine proportions and rate per minute of questions that occurred in a theory physical education (PE) lesson. The study involved a class of 38 students of a College of Education and one teacher (intern) of a University in Ghana. The PE lesson was video-recorded. Self-developed event recording instrument was used to collect data on questioning behaviours of both the teacher and students. Data collection instrument was validated by an expert in research (senior university faculty). Descriptively, data were analysed and presented in tables and figures. Intra-observer results of classroom events relating to questioning behaviours of the teacher and students were evaluated. Findings revealed that the proportion of lower-order questions was generally larger than higher-order questions that occurred during the lesson. Findings also showed that a total of 36 questions were asked by the teacher during the 35 minutes interaction. Specifically, the rate per minute of questions asked by the teacher at various question levels was 0.38 for knowledge, 0.35 for understanding, 0.2 for synthesis, 0.25 for evaluation, and zero for application and analysis questions. No question was asked by any of the students in the entire 35 lesson duration. We recommend that teachers should make conscious efforts to create opportunities for learners to ask questions in every teaching and learning environment.

Remote Flipped Teaching during the COVID‐19 Pandemic

Flipped teaching (FT) is known to foster active learning and increase student engagement. The flipped classroom design organizes learning in such a way that the lower‐order thinking such as memorization is expected at the individual student level in the form of homework and the higher‐order thinking including application and analysis occurs in the classroom in the presence of the instructor. With the sudden onset of the COVID‐19 pandemic, instructors and students were given no time to adjust to the remote educational set up. Flipped teaching, like most other instructional modalities, had to be transformed into an online platform. The present study assessed the effectiveness of FT in a remote setting in a graduate‐level physiology course (n=33). The course content included lecture videos, slides, and reading assignments which were made available on BlackBoard, the Learning Management System. Students were expected to have reviewed the assigned content prior to the virtual synchronous sessions. During the synchronous meeting, a question‐answer (Q&A) session allowed students to review the content yet again. Followed by the Q&A session, an individual quiz, containing lower‐order questions, was given to assess student preparation. Soon after the review of the quiz, students in small groups (4‐5 students per group) were allowed to use breakout rooms to complete a set of questions which were designed at the higher order of Bloom's taxonomy and expected students to apply knowledge, analyze or synthesize their collaborative responses. The answers to these questions were reviewed during the first part of the following synchronous session where students were selected randomly to answer these questions which prompted them to come prepared. Student surveys were administered at the beginning and the end of the course to assess their knowledge gain, perception of the remote FT format, and confidence in completing the course. Based on the survey results, it was found that the students adjusted well to the remote course setting (p<0.001) and their confidence in completing the course successfully increased from the beginning to the end of the semester (p<0.05). Students also found that the collaborative activities (93.55%) and the in‐class discussions (96.77%) were effective and were well adjusted to the synchronous computerized exams (90.32%) and the individual in‐class assessments (80.65%). Overall, the remote FT method was successful and therefore, the principles behind FT could be employed in virtual teaching.

AN EVALUATION OF WH-QUESTION IN ENGLISH IN MIND TEXTBOOK AN BRIGHT AN ENGLISH TEXTBOOK SEEN IN TERMS OF BLOOM’S NEW TAXONOMY

The study aimed at identifying and analyzing the types and levels of WH-question in two English Textbooks namely, English in Mind Textbook (EIM) and Bright An English Textbook (BAE) for eighth grade junior high school. The objective of the analysis was to evaluate the WH –questions in the light of the Bloom’s New Taxonomy (1956) Taxonomy learning objectives ( Remembering, Understanding, Applying, Analyzing, Evaluating, Creating).Content analysis was used to conducted all the WH-questions in the two textbooks. All the WH-questions were collected, listed, and analyzed according to Bloom’s New Taxonomy. The result from a total of 140 questions (70 questions in EIM and 70 questions in BAE) showed that most of the questions were align with remembering , understanding, and applying as the three lower – level categories, while analyzing, evaluating, and creating as the three higher-level categories constitude the lowest frequency in the two textbooks.Another result also for the proportion of higher-order questions in both Textbook showed that EIM received higher distribution while BAE constitute the highest distribution on lower-order questions. The result indicate that the textbooks failed to engage learners in the questions requiring higher levels of cognitive learning objectives.In the light of the results, the researcher suggested to modified the questions in the textbooks to cover the six levels of Bloom’s new Taxonomy and to train teachers and author of textbook to use and write question following the Bloom’s new Taxonomy. Keywords : Bloom’s New Taxonomy, WH-questions, content analysis, English Textbook

Teachers' Questioning: A Survey of Nepali Schools

This study intends to investigate the present situation of teachers' questioning in Nepali school, test the use of questions types, and analyze across the characteristics of the teachers. The data were collected from a survey of 92 teachers of 81 schools including both community and institutional schools. The questionnaire was prepared largely based on Introduction of Core Skills for Teachers (British Council, 2016). The Cronbach's alpha was in the range of 0.7 that ensured the reliability of the questionnaire. The descriptive statistics and inferential statistics were employed to find out the results. The results show that in overall study, as the effective teachers, Nepali school teachers use lower-order questions, higher-order and follow-up questions in a continuum of frequency from lower to higher order irrespective of teachers' selected characteristics. However, the teachers with a few years of experience use lower-order questions, and trained teachers use more frequently follow-up questions to a significant extent. Based on the findings of this study, Nepali school policy makers and leaders can adapt the existing teacher development programs and policies to make effective teaching and learning. Moreover, teachers can also get insights actually at what level they are in questioning techniques, and further, improve their questioning techniques.&#x0D; Int. J. Soc. Sc. Manage. Vol. 8, Issue-1: 333-340

Higher-Order Assessment in Gross Anatomy: A Comparison of Performance on Higher- Versus Lower-Order Anatomy Questions between Undergraduate and First-Year Medical Students.

Gross anatomy is considered by many the backbone of medical education. While learning anatomy has a reputation of requiring mainly rote memorization, modern day anatomy education often involves instruction and assessment at cognitive levels that foster higher-order thinking. In many instances, these higher-order anatomical concepts are taught to graduate students in healthcare-related fields, such as medicine. At this level, students are expected to apply and analyze anatomical information since that is what will ultimately be expected of them as professionals. In contrast, undergraduate anatomy education is typically more introductory in nature and often takes place in the setting of a large-enrollment course that serves as a prerequisite for many health sciences degree programs. In this study, variables related to the assessment of higher-order concepts in clinical anatomy were compared between first-year medical students and undergraduate students enrolled in an upper-level human gross anatomy course. Results demonstrate that undergraduate students perform lower than medical students overall, but the degree of difference in how they perform on higher- versus lower-order questions is comparable. The most notable exception is on practical examinations, where undergraduate students tend to struggle more with applying and analyzing information. Exploration of additional variables provides insight into how the cognitive level being assessed affects the time it takes to answer a question and how different practical examination question types and formats influence student performance. Findings presented in this study have implications for designing anatomy courses and underscore the importance of blueprinting assessments.

THE PRACTICE AND CHALLENGES OF IMPLEMENTING CRITICAL THINKING SKILLS IN EFL TEACHERS QUESTIONING BEHAVIOR

The development of critical thinking is closely related to the questions asked by teachers as the teachers higher-order questions are cognitively demanding to promote students critical thinking. The purposes of this study are to investigate how critical thinking was implemented in EFL teachers questions in the classroom and the challenges faced by EFL teachers in integrating critical thinking questions. In addition, this study also investigated students reasons for non-response to particular questions asked by teachers. Applying mixed-method research design, the data were collected from classroom observation, interview, questionnaire and documentations. The research participants were two English teachers and 229 number of students of year 8. The results showed that the most dominant questions asked by the two English teachers were in Lower-order questions (78,8%), while the Higher-order questions were only (21,2%) questions. Second, the challenges faced by teachers were students intelligence has yet to reach the level of higher-order thinking, unconducive-classroom environment, teachers creativity itself, parental involvement and students lack of vocabulary. Third, dealing with the reason for students non-response in particular questions asked by teachers, there were some reasons, such as students are afraid of making mistakes, they could not put ideas into words, and lack of vocabularies. In conclusion, the teachers faced various challenges in implementing critical thinking questions in the classroom.

Comparison of level of cognitive process between case-based items and non-case-based items of the interuniversity progress test of medicine in the Netherlands.

PurposeIt is assumed that case-based questions require higher-order cognitive processing, whereas questions that are not case-based require lower-order cognitive processing. In this study, we investigated to what extent case-based and non-case-based questions followed this assumption based on Bloom’s taxonomy.MethodsIn this article, 4,800 questions from the Interuniversity Progress Test of Medicine were classified based on whether they were case-based and on the level of Bloom’s taxonomy that they involved. Lower-order questions require students to remember or/and have a basic understanding of knowledge. Higher-order questions require students to apply, analyze, or/and evaluate. The phi coefficient was calculated to investigate the relationship between whether questions were case-based and the required level of cognitive processing.ResultsOur results demonstrated that 98.1% of case-based questions required higher-level cognitive processing. Of the non-case-based questions, 33.7% required higher-level cognitive processing. The phi coefficient demonstrated a significant, but moderate correlation between the presence of a patient case in a question and its required level of cognitive processing (phi coefficient= 0.55, P< 0.001).ConclusionMedical instructors should be aware of the association between item format (case-based versus non-case-based) and the cognitive processes they elicit in order to meet the desired balance in a test, taking the learning objectives and the test difficulty into account.

The effect of first and second language use on question types in English medium instruction science classrooms in Hong Kong

ABSTRACTThe role of the first language (L1) in second language (L2) learning has been widely discussed and researched in the second Language Acquisition (SLA) literature. Benefits of L1 use have been found for vocabulary acquisition, and in facilitating comprehension thereby alleviating student anxiety. Many teachers appear to consider L1 use as a necessary option in L2 pedagogy. Disadvantages of L1 use are argued to be that it reduces exposure to the L2 thereby not enabling more implicit forms of L2 acquisition. Considerably less research has been carried out in English medium instruction (EMI) classrooms, surprisingly, as these are predicated on the benefits of massive exposure to the L2. This study investigated the effect of L1 and L2 use on teacher question types and interaction patterns in science lessons in early EMI and late EMI schools in Hong Kong. Our findings show that in late EMI, where there was much greater use of the L1, teachers tended to use higher order questions and were more interactive. In early EMI, despite students having received four more years of greater exposure to English instruction than late EMI students through the use of English as the vehicular language, teachers asked lower order questions and the interaction was more monologic. The implications for policy and pedagogy are discussed.

Pushing Critical Thinking Skills With Multiple-Choice Questions: Does Bloom's Taxonomy Work?

Medical school assessments should foster the development of higher-order thinking skills to support clinical reasoning and a solid foundation of knowledge. Multiple-choice questions (MCQs) are commonly used to assess student learning, and well-written MCQs can support learner engagement in higher levels of cognitive reasoning such as application or synthesis of knowledge. Bloom's taxonomy has been used to identify MCQs that assess students' critical thinking skills, with evidence suggesting that higher-order MCQs support a deeper conceptual understanding of scientific process skills. Similarly, clinical practice also requires learners to develop higher-order thinking skills that include all of Bloom's levels. Faculty question writers and examinees may approach the same material differently based on varying levels of knowledge and expertise, and these differences can influence the cognitive levels being measured by MCQs. Consequently, faculty question writers may perceive that certain MCQs require higher-order thinking skills to process the question, whereas examinees may only need to employ lower-order thinking skills to render a correct response. Likewise, seemingly lower-order questions may actually require higher-order thinking skills to respond correctly. In this Perspective, the authors describe some of the cognitive processes examinees use to respond to MCQs. The authors propose that various factors affect both the question writer and examinee's interaction with test material and subsequent cognitive processes necessary to answer a question.

Off-task multitasking, note-taking and lower- and higher-order classroom learning

We examined whether multitasking via concurrent off-task text messaging during an academic presentation impacted students’ performance on tests assessing lower-order and higher-order learning. College students (N = 183) were assigned to one of two conditions involving either concurrent texting or not texting during an academic presentation, or to a no presentation condition. Students in presentation conditions were encouraged to take hand-written notes. Between-participants analyses revealed that students who saw the presentation performed better on learning measures than the control group who did not see the presentation, indicating that students did learn from the presentations. Non-texters scored significantly higher than texters on multiple choice tests of factual, lower-order information (e.g., knowledge, comprehension), but not on essays requiring higher-order application, analysis, evaluation, and synthesis of information. Within-participants analyses demonstrated that texters performed more poorly on lower-order questions that were based on information presented at times when they were texting. Non-texters took more quality notes than texters; amount of quality notes was positively related to test scores of all types. The amount of quality notes taken partially mediated the relationship between texting condition and multiple choice test scores. It appears that multitasking with media devices during an academic presentation interferes with note-taking and the encoding of information specific to the presentation.

An Investigation of Critical Thinking Manifested in the Questions of EFL Textbooks for Tertiary-Level English Majors of China

This study aimed at investigating to what extent critical thinking (CT) is manifested in the questions of three English textbooks commonly used in Intensive Reading course for undergraduate English majors of China. For this purpose, the after-text reading comprehension questions in English Through Culture, Contemporary College English and Think English were coded and classified based on Bloom et al. (1956). Two major findings have emerged. First, as far as the simple questions are concerned, the lower-order questions are more prevalent in the EFL textbooks, but a gradual increase in compilers’ awareness of incorporating CT cultivation into textbooks is observed over time, with higher-order questions outnumbering their counterparts in Think English. As regards the composite questions, Think English has the highest percentages of upward questions, wavy questions and upward waveform questions, three types of composite questions most beneficial to CT cultivation. Therefore, the recently published Think English satisfactorily fulfills the educational objective of fostering English majors’ CT skills in terms of its questioning design. The study concludes by discussing its implications and suggestions for future avenues of research.

Why Would You Say Goodnight to the Moon? Response of Young Intellectually Gifted Children to Lower and Higher Order Questions During Storybook Reading

Research into the effect of questions asked during storybook reading in preschool settings has rarely investigated questions that elicit higher order thinking. In the current study, Blank et al.’s Four Levels of Abstraction were used to code teacher questions and child responses from 177 individual storybook reading sessions with eight intellectually gifted 3- and 4-year-old children. The aim was to investigate the level of cognitive response gifted preschoolers made to lower and higher order questions during shared book reading. As expected, lower order questions, the most frequent form of teacher questions, elicited mainly lower order responses. Significant cognitive correspondence was also found for higher order questions, which elicited higher order child responses 88% of the time. This suggests that higher order questioning would be a valuable addition to preschool storybook reading, particularly to extend the thinking of young intellectually gifted children.