Multiple-choice Comprehension Questions Research Articles

Enabling text comprehensibility assessment for people with intellectual disabilities using a mobile application

In research on Easy Language and automatic text simplification, it is imperative to evaluate the comprehensibility of texts by presenting them to target users and assessing their level of comprehension. Target readers often include people with intellectual or other disabilities, which renders conducting experiments more challenging and time-consuming. In this paper, we introduce Okra, an openly available touchscreen-based application to facilitate the inclusion of people with disabilities in studies of text comprehensibility. It implements several tasks related to reading comprehension and cognition and its user interface is optimized toward the needs of people with intellectual disabilities (IDs). We used Okra in a study with 16 participants with IDs and tested for effects of modality, comparing reading comprehension results when texts are read on paper and on an iPad. We found no evidence of such an effect on multiple-choice comprehension questions and perceived difficulty ratings, but reading time was significantly longer on paper. We also tested the feasibility of assessing cognitive skill levels of participants in Okra, and discuss problems and possible improvements. We will continue development of the application and use it for evaluating automatic text simplification systems in the future.

Impact of text-to-speech features on the reading comprehension of children with reading and language difficulties.

This study investigated the reading comprehension scores of students with reading and language difficulties after reading a passage with and without text-to-speech (TTS). Students, ages 8 to 12years, read five passages under the following conditions: (a) silent read, (b) read aloud, (c) listen only, (d) TTS with no highlighting, and (e) TTS with highlighting. Students answered multiple-choice comprehension questions following each condition. Mixed ANOVAs were performed to determine whether TTS improved reading comprehension. TTS significantly improved comprehension in comparison to no TTS, and specifically, TTS with no highlighting and TTS with highlighting resulted in significantly higher comprehension scores compared to silent read. No other significant differences were found across conditions including between the presentational features of TTS, specifically TTS with no highlighting and TTS with highlighting conditions. Students were grouped as dyslexia only or reading and language impairment based on their test results. Findings suggested that students with dyslexia only scored significantly higher on reading comprehension questions in all reading conditions and derived significantly more benefit in reading comprehension from TTS and the listen only condition compared to students with Reading and Language Impairment. Overall, TTS may be a helpful tool for supporting the reading comprehension of students with reading and language difficulties, particularly for students with dyslexia only; however, further studies are needed to explore the benefits of TTS' presentational features such as highlighting with students with reading and language difficulties.

Comprehension, Diagram Analysis, Integration, and Interest: A Cross-Sectional Analysis

The current study examines a multidimensional set of outcome variables to understand whether different pre-reading scaffolds influence students’ text comprehension, diagram analysis, text integration, and interest; and investigates these constructs cross-sectionally to identify any progression as students move across grades. One-hundred fifty-six 3rd through 6th grade students enrolled in a public laboratory school were randomly assigned to one of three pre-reading conditions intended to activate or build students’ topic knowledge. Students completed a series of before, during, and after reading activities while engaging with grade appropriate texts about the topics of ecosystems and living things. Results indicate that there were no significant differences between the three pre-reading conditions on any of the four constructs of interest. Students across grade levels performed well on multiple-choice comprehension questions, but not as well on diagram analysis questions or an open-ended integration task. Implications and future directions are discussed.

Dutch students’ evaluations of EMI and L1MOI lectures: The role of non-native pronunciation

In a within-subjects experiment, 119 Dutch listeners evaluated authentic English and Dutch lecture fragments recorded by the same Dutch lecturer. For each fragment, they answered multiple-choice comprehension questions and rated the lecturer on comprehensibility, attitudinal measures (status, competence, likeability and dynamism), and perceived teaching quality. Findings showed that actual comprehension was not different for the English and the Dutch fragments. The lecturer was evaluated less positively in the English fragments than in the Dutch fragments in terms of status, competence, dynamism and teaching quality, but not likeability. A mediation analysis showed that language choice had no direct effect but only indirect effects on perceived teaching quality through comprehensibility and non-native pronunciation. Our study shows that the use of EMI instead of L1MOI would not appear to result in reduced comprehension but does result in more negative perceptions of the lecturer, and that non-native pronunciation contributes to these perceptions.

Comprehension processes on question-answering activities: A think-aloud study

Introduction. Students often answer questions from available expository texts for assessment and learning purposes. These activities require readers to activate not only meaning-making processes (e.g., paraphrases or elaborations), but also metacognitive operations (e.g., monitoring readers’ own comprehension or self-regulating reading behaviors) in order to successfully use textual information to meet the task demands. The aim of the study was to explore the meaning-making processes readers activate while answering questions and using textual information at hand, and how they monitor and control the processing of these tasks. Method. Forty eighth graders read two expository texts and answered ten multiple-choice comprehension questions per text. For each question, participants were forced to select the relevant pieces of textual information to provide the correct answer. Further, participants thought aloud in one of the texts, whereas they performed the task in silence in the other. This reading scenario served to collect valuable data on the reader’s question-answering process. The task was administered individually on a computer-based environment, which allowed recording the students’ reading behavior online. Results. Results showed processing differences between the two recursive steps of question-answering. While reading the question readers mainly restated information and focused on monitoring the comprehension of the questions and their correspondence with the alternatives. However, they tended to paraphrase and assess textual relevance when searching the text. The study also reveals that think-aloud methodology affected these processes and outcomes differently. Discussion and Conclusion. This study advances on our knowledge of cognitive and metacognitive processes involved in answering questions from an available text, a task extensively used to measure reading literacy skills as well as for learning purposes in academic settings. It shows how readers activate meaning-making processes while reading questions and searching the text for task-relevant information, and how they monitor the question-answering process. The study also reveals that think-aloud methodology may affect these processes differently. Apart from advancing our theoretical knowledge, our study also has important practical applications for the assessment of reading literacy skills.

The use of generalizability theory in investigating the score dependability of classroom-based L2 reading assessment

Among the variety of selected response formats used in L2 reading assessment, multiple-choice (MC) is the most commonly adopted, primarily due to its efficiency and objectiveness. Given the impact of assessment results on teaching and learning, it is necessary to investigate the degree to which the MC format reliably measures learners’ L2 reading comprehension in the classroom context. While researchers have claimed that the longer the reading test (i.e., more test items and passages), the higher its overall reliability, few studies have investigated the optimal number of items and passages required for reliable classroom-based L2 reading assessment. To address this research gap, I adopted generalizability (G) theory to investigate the score reliability of the MC format in classroom-based L2 reading tests. A total of 108 ESL students at an American college completed an English reading test that included four passages, each of which was accompanied by five MC comprehension questions. The results showed that the score reliability of the L2 reading test was critically influenced by the number of items and passages, inasmuch as a different combination of the number of passages and items altered the degree of reliability. Implications for practitioners and educational researchers are discussed.

ATTENTION TO FORM AND MEANING REVISITED

AbstractMotivated by a series of interconnected studies on simultaneous attention to form and meaning, we revisit L2 learners’ real-time processing of text by using eye-tracking as an unobtrusive method to provide concurrent data on attention allocation. Seventy-five L2 Spanish learners were instructed to attend to an assigned form in a reading passage and to press a button when they noticed it. After reading the passage, the learners answered 10 multiple-choice comprehension questions. The participants’ responses to the comprehension questions and their reading behaviors reflected in eye-movement data suggest that attention to grammatical form may hinder L2 learners’ simultaneous attention to form and meaning. However, individual differences in global text processing contributed to the differences in the participants’ text-comprehension scores over and above the task instruction to attend to form: Slower L2 readers who read the passage more carefully showed better text comprehension.

Dual Modality Effects on College Students’ Comprehension of Short Texts during a Simulated Self-Study Session

College students enrolled in a reading support course were asked to (a) read a short text, (b) listen to a second text, and (c) read + listen to a third text and answer multiple-choice comprehension questions about each text. Each condition employed a self-study format allowing for constant availability of text input and extra time to revisit text content for further study. Contrary to expectations that students’ comprehension of texts would benefit from the dual modality presentation of the text in the read + listen combo condition, this condition produced the lowest comprehension scores, suggesting our students might have experienced interference from dual modality processing of texts even when they had extra time to revisit text input. Further revealed were indications of listening comprehension possibly setting the upper limit of our sample’s comprehension capacity. Limitations and implications for college teaching research and practice are discussed.

What Eye Movements Reveal About Later Comprehension of Long Connected Texts.

We know that reading involves coordination between textual characteristics and visual attention, but research linking eye movements during reading and comprehension assessed after reading is surprisingly limited, especially for reading long connected texts. We tested two competing possibilities: (a) the weak association hypothesis: Links between eye movements and comprehension are weak and short-lived, versus (b) the strong association hypothesis: The two are robustly linked, even after a delay. Using a predictive modeling approach, we trained regression models to predict comprehension scores from global eye movement features, using participant-level cross-validation to ensure that the models generalize across participants. We used data from three studies in which readers (Ns=104, 130, 147) answered multiple-choice comprehension questions ~30min after reading a 6,500-word text, or after reading up to eight 1,000-word texts. The models generated accurate predictions of participants' text comprehension scores (correlations between observed and predicted comprehension: 0.384, 0.362, 0.372, ps<.001), in line with the strong association hypothesis. We found that making more, but shorter fixations, consistently predicted comprehension across all studies. Furthermore, models trained on one study's data could successfully predict comprehension on the others, suggesting generalizability across studies. Collectively, these findings suggest that there is a robust link between eye movements and subsequent comprehension of a long connected text, thereby connecting theories of low-level eye movements with those of higher order text processing during reading.

Everyday reading in aphasia: Does advance picture context influence reading speed and comprehension?

ABSTRACT Background Both reading speed and comprehension at the text level are often impaired in people with aphasia, potentially impacting their daily life. Provision of advance context (e.g., pictures provided to the reader prior to reading) is found to facilitate text processing and comprehension in typical readers. However, there has been no investigation of whether advance context influences reading in people with aphasia, although some studies have found no benefit on people with aphasia’s comprehension accuracy from contextual supports presented simultaneously with text. It also remains unclear whether context may impact people with aphasia’s understanding of any particular type of textual information (main ideas vs details, stated ideas vs implied ideas). Aims This study aimed to investigate the effect of advance picture context on reading speed, overall comprehension accuracy and comprehension of different types of ideas for naturalistic everyday texts in people with aphasia. Methods & Procedures Twenty-nine adult typical readers and 10 people with aphasia with varying reading ability took part in the study. Participants read three matched sets of short news articles which were either preceded by a related picture, no picture, or (as fillers) an unrelated picture and were followed by multiple-choice comprehension questions. For each text, the questions assessed the understanding of a stated main idea, an implied main idea, a stated detail, and an implied detail. Mixed-effects regression analyses were performed to examine the effect of advance (related) picture context on participants’ reading time and comprehension accuracy. Outcomes & Results The results showed a significant effect of advance picture context on reading speed, with both typical readers and readers with aphasia showing significantly shorter reading times when a picture was available. Advance context did not affect overall comprehension accuracy or accuracy for any subtype of textual information for either group of participants. Conclusion Advance context facilitates text processing in people with aphasia, resulting in increased reading speed for everyday texts, although it does not influence their comprehension accuracy for these types of texts. This suggests that therapeutic approaches could develop more effective use of advance contextual organisers in order to increase people with aphasia’s reading efficiency and ultimately, their confidence and pleasure in reading.

Linking Test-Taking Process to Performance Through Mixed-Effects Regression Models: A Response Process–Based Validation Study

Answering the call for response process–based validation, this study shows how researchers can evaluate validity evidence of test scores based on the mental “processes” test-takers use, rather than based on correlations with other “outcome” measures. The proposed methods for process-based validation studies are demonstrated using a sample of 189 adults who took two listening comprehension tasks. Immediately after completing each task, the test-takers filled out a 10-item survey to reflect on the mental processes involved in reaching their answers. These 10 process variables attempted to capture five desired and five undesired response processes in answering multiple-choice listening comprehension questions. We investigated the relationships between these process variables and the binary outcome of item score (correct vs. incorrect) using mixed-effects logistic regression models, and showed how the results could provide validity evidence (or lack thereof). By doing so, we offer an alternative approach to study response process and test performance and encourage more process-based validation studies.

Comprehension in interpreting and translation: testing the phonological interference hypothesis

ABSTRACTStudies on the comprehension process in interpreting have shown that concurrent processing reduces recall in simultaneous interpreting. This effect has been attributed to phonological interference: since the articulatory loop is busy with the parallel vocalization of two streams of speech, encoding is impaired to such an extent that interpreters are not able to remember much of what they have just interpreted. On the other hand, recent studies on the translation process show that comprehension and production overlap in written translation in a way that is similar to simultaneous interpreting. Therefore, this article examines the role of phonological interference in written and oral translation to determine whether or not it also hinders recall in written translation, and to gauge how task requirements affect the comprehension process in translation and interpreting. In this study, comprehension was measured through summarization, multiple choice comprehension questions and cloze questions administered after simultaneous interpretation and translation activities were completed by a group of advanced interpreting students. Results suggest that both translation and interpreting share similar features regarding parallel processing and, furthermore, that the process of comprehension is influenced by the demands associated with translation and simultaneous interpreting.

The Role of Short Vowels in Arabic Listening Comprehension.

The goal of the present study was to investigate the effect of short Arabic vowels on listening comprehension. It is well documented that short vowels affect reading and reading comprehension in Arabic literacy. Since reading comprehension and listening comprehension share many commonalities, it is assumed that short vowels will positively affect listening comprehension too. Two listening conditions were presented to students in three grade levels, 7th, 10th and 12th grades: a fully vowelized text and the second without short vowels (on ends of words). Students had to listen and answer multiple choice comprehension questions about the read aloud texts. Further, interviews were conducted with 10 volunteered students from each grade level to get their feedback about their experience. The results indicated significant differences between the two listening conditions in favor of the fully vowelized condition. The results are discussed in relation to different orthographies, reading comprehension, phonology, working memory and comprehension.

Reading in English as a Foreign language: Examining differences in reading speed, comprehension, efficacy and L1 cross-linguistic influence across grades

Reading in a second or foreign language (L2) is a more complicated process than reading in one's first language because it requires additional demands on the reader. Reading can be assessed from various domains. Reading fluency is known to be the most determining domain to identify reading problems (Rasinski, 2000). However, the goal of reading is comprehending a text, so comprehension should not be disregarded either. Despite the importance of reading in an L2, few studies have focused on examining its effects. The goal of the present study is threefold: first, to provide the estimates of L2 English reading rates of L1 Catalan/Spanish students of different ages and grades; second, to examine whether there are significant differences between participants of different grades in terms of reading rates, comprehension and efficacy; and third, to explore whether the students' reading rates in the L1 are correlated with their L2 reading rates and if so, the extent to which L1 reading rates explain L2 reading rates.&#x0D; 790 Catalan/Spanish learners of English participated in the present study. Participants were from 5th grade primary (age 12) to 2nd baccalaureate (age 18). They were asked to read two texts, one in English and the other one in their L1 (they could choose between Catalan or Spanish). They were asked to answer 7 multiple-choice comprehension questions after reading each text. Next, their scores on fluency (words read per minute), comprehension (percentage of comprehended text) and efficacy (a formula that integrates speed and comprehension) were obtained. Results show that reading fluency and comprehension are not linear and that they do not necessarily improve over time. Results also show that L1 and L2 reading fluency rates are highly correlated and that L1 reading rates explain a large percentage of L2 reading rates.

Reading in English as a Foreign language: Examining differences in reading speed, comprehension, efficacy and L1 cross-linguistic influence across grades

Reading in a second or foreign language (L2) is a more complicated process than reading in one's first language because it requires additional demands on the reader. Reading can be assessed from various domains. Reading fluency is known to be the most determining domain to identify reading problems (Rasinski, 2000). However, the goal of reading is comprehending a text, so comprehension should not be disregarded either. Despite the importance of reading in an L2, few studies have focused on examining its effects. The goal of the present study is threefold: first, to provide the estimates of L2 English reading rates of L1 Catalan/Spanish students of different ages and grades; second, to examine whether there are significant differences between participants of different grades in terms of reading rates, comprehension and efficacy; and third, to explore whether the students' reading rates in the L1 are correlated with their L2 reading rates and if so, the extent to which L1 reading rates explain L2 reading rates.&#x0D; 790 Catalan/Spanish learners of English participated in the present study. Participants were from 5th grade primary (age 12) to 2nd baccalaureate (age 18). They were asked to read two texts, one in English and the other one in their L1 (they could choose between Catalan or Spanish). They were asked to answer 7 multiple-choice comprehension questions after reading each text. Next, their scores on fluency (words read per minute), comprehension (percentage of comprehended text) and efficacy (a formula that integrates speed and comprehension) were obtained. Results show that reading fluency and comprehension are not linear and that they do not necessarily improve over time. Results also show that L1 and L2 reading fluency rates are highly correlated and that L1 reading rates explain a large percentage of L2 reading rates.

Do Consumers Understand PCP Car Finance? An Experimental Investigation

Personal contract purchase (PCP) plans are innovative and increasingly popular forms of car finance. PCPs are inherently more complex than established financing options. The present study used experimental behavioural science to explore consumers’ comprehension of PCP plans and scope for beneficial interventions. Choice tasks, product rating tasks, and multiple choice comprehension questions were deployed to measure the consistency of decision-making and explicit comprehension of the product. Disclosures and advice were varied across conditions. A representative sample (n = 100) of consumers was initially given information on PCP deals as typically disclosed by car dealers. Results revealed that understanding was poor. One quarter of participants performed below chance on multiple-choice comprehension questions. Participants were prone to inconsistencies and objective mistakes when deciding between and rating offers. Disclosures designed to improve processing of mileage and cost information had ambiguous effects. Consumer advice sheets improved comprehension and reduced mistakes, with advice containing a diagram outperforming advice containing only text. The findings raise consumer protection concerns and support improved advice and stronger regulation.

I must have missed that: Alpha-band oscillations track attention to spoken language

Attention is critical to the construction of mental representations of language context during comprehension. We investigated the consequences of momentary lapses in attention during listening comprehension on neural activity and behavior. Participants listened to two full-length stories while EEG was recorded, and afterwards completed multiple choice comprehension questions. Listening was periodically interrupted by attention probes, in which participants were asked whether their attention immediately preceding the probe's appearance was focused on the story. The results showed that (1) participants spent a substantial amount of time off-task, endorsing attention lapses on over 30% of probes; (2) for probes on which an attention lapse was endorsed, later accuracy on comprehension questions querying pre-probe information was decreased; (3) the pre-probe period just before the endorsement of an attention lapse was characterized by a greater percentage of above-threshold oscillations in the alpha-band (8–12 Hz) compared to just prior to the endorsement of on-task or split-attention listening; and (4) when participants made “I have no idea” responses to comprehension questions, their EEG record revealed a greater percentage of above-threshold alpha oscillations during the original presentation of the information queried by the comprehension questions, compared to correct responses or incorrect guesses. These results connect changes in neural activity in the alpha band to episodes of mind-wandering during listening comprehension, and in turn to decreased comprehension accuracy. This demonstrates how alpha can be used to track attentional engagement during language comprehension, and illustrates the dependence of successful language comprehension on attention.

The Impact of Dysphonic Voices on Children's Comprehension of Spoken Language

This study investigated the effect of teachers' dysphonic voices on children's listening comprehension. One hundred thirty-four grade three and four students were recruited from local primary schools in Hong Kong. They were required to listen to six passages, three in Cantonese and three in English, which were either read in normal, mildly dysphonic, or severely dysphonic voices. The students were required to complete six multiple-choice comprehension questions upon listening to each passage. Comprehension performance across languages, dysphonic severities, genders, and question types were examined. The results showed that listening comprehension was significantly poorer even when speaker's voice quality was mildly impaired. Performance in Cantonese was generally better than that in English but no significant difference in the pattern of decline was found. Both boys and girls suffered to similar extent under dysphonic situations. Differences in performance in various question types were discussed. These findings support the urgent need to implement voice care education for the teaching profession.

Evaluating Knowledge of Business Processes

1 IntroductionThis paper introduces our research on testing methods that can be employed in testing the knowledge of business processes. This study approaches the research question: Which method of testing the knowledge about processes/procedures is more suited for evaluating the capacity of employees to execute them? Typically, process knowledge is evaluated by asking multiple-choice or open questions on the process documentation. Most organizations document some of the main processes in textual form and then struggle to keep it up to date. A growing number of organizations acknowledge the importance of business processes, document them using some form of graphical models, and employ information systems tailored to support them. However, there are still many organizations that haven't documented processes. In these organizations, employees learn by training and/or experience how to achieve organizational goals. No matter if process documentation exists or not, it is critical for managers to be able to evaluate how well the employees are able to execute processes.Shallow knowledge about processes relates to learning the main steps to be performed, their sequence, the documents and data involved, etc. A deeper understanding resides in, for example, issues like contingency steps in case of errors; overview of organization-wide processes, etc. Current testing based on multiplechoice questionnaires don't go beyond the shallow understanding. We argue that a new testing approach is needed. It should aim to put the tested employee in the position where problem-solving deep knowledge is needed, rather than the ability to memorize process steps. In this paper we introduce a first take on this challenge. We evaluate if asking questions in a graphical -interactive manner is better than the multiple-choice way. Better is interpreted in terms of a greater number of correctly asked questions as well as the time needed to answer comprehension questions. The context of our experiment is an organization of higher education. More specifically, we use the annual admission process to evaluate how accurate different types of participants know and understand the entire process, and if they are able to execute it in any specific case that might arise.The paper unfolds as follows. First, we provide an overview of the theoretical foundations by reviewing papers related to factors influencing model understanding and experimental design. In the next section, we provide the details of our controlled experiment. The single factor of the experiment is the comprehension question presentation format. In section 4 we introduce the results and our data analysis. We end with conclusions and the implications of our findings.2Related WorkBusiness process models are key artefacts in the development of information systems. While one of their main purposes is to facilitate communication among stakeholders, little is known about the factors that influence their comprehension by human agents. To date, the body of research on process model understanding relies on controlled experiments based on multiple-choice comprehension questions. Therefore, in this section, we approach two main related research avenues: one on process model understanding, and a second one on controlled experiments in process model settings.The notation that we use in our experiment is the Business Process Model and Notation (BPMN) which is the industry standard in process modelling. It has straight-forward syntax and semantics. So-called connectors (XOR, AND) define complex routing constraints of splits (multiple outgoing arcs) and joins or merges (multiple ingoing arcs). Through XOR, when splitting, the sequence flows to exactly one of the outgoing branches. When merging, it awaits one incoming branch to complete before triggering the outgoing flow. Through AND, when used to split the sequence flow, all outgoing branches are activated simultaneously. When merging parallel branches process flow waits for all incoming branches to complete before triggering the outgoing flow. …

Improving academic learning from computer-based narrative games

Although many strong claims are made for the power of computer games to promote academic learning, the narrative content of a game may reduce the learner's tendency to reflect on its academic content. The present study examines adding a low-cost instructional feature intended to promote appropriate cognitive processing of the academic content during play. College students played a computer adventure game in which they guided a character through a bunker in search of lost artwork, building electromechanical devices to open stuck doors along the way. In Experiment 1, students who filled out worksheets about wet-cell batteries before and during the game outperformed students who played the game without worksheets on a written explanation of how wet-cell batteries work (d = 0.92), multiple-choice comprehension questions about wet-cell batteries (d = 0.67), and open-ended transfer problems about wet-cell batteries (d = 0.74). In Experiment 2, participants who completed only the in-game worksheet outperformed the control group on a written explanation of wet-cell batteries (d = 0.59) and transfer problems (d = 0.67), whereas participants who completed only the pre-game worksheet did not outperform the control group on any measure. These findings point to the learning benefits of adding instructional features suggested by cognitive theories of learning.