Think-aloud Protocol Data Research Articles

Mixed reality environment for learning sensing technology applications in Construction: A usability study

Mixed reality is gradually becoming ubiquitous and significant in education owning to the inherent benefits of active participation and tacit knowledge development in a safe and engaging environment. However, limited studies have explored design features that facilitate its use as a pedagogical tool in construction education, particularly in equipping students with experiential skills that are otherwise challenging to obtain due to resource constraints and limited access to construction sites. By evaluating eye tracking, usability questions, and think-aloud protocol data and verbal feedback, this study investigated the usability of a mixed reality environment designed for equipping construction engineering students with competencies for deploying sensing technologies on construction projects. Results revealed features such as accuracy of represented construction activities, quality of animations, and easy access to information and resources as important for designing efficient mixed reality learning environments. While the usability data suggested that the user interface was usable, eye tracking provided profundity on encountered difficulties with the user interface. Through concise outline and sequential design of the user interface, this study revealed that knowledge scaffolding can improve task performance in a mixed reality learning environment. This study adds value to existing literature, in particular by providing insights into the affordances of mixed reality environments that address the technological gap between the construction industry and construction engineering education. The mixed reality learning environment contributes to the cognitive apprenticeship theory through the use of game objects to develop procedural knowledge for addressing construction industry challenges with sensing technologies.

Remodeling writers’ composing processes: Implications for writing assessment

Reaching a better understanding of the composing process is essential to improving writing assessment practices. This paper reports a remodeling attempt of written text production processes. The author argues for remodeling writers’ cognitive processes through reviewing the changing conceptualizations in six notable global writing process models (Chenoweth and Hayes, 2001, Flower & Hayes, 1981, Hayes, 1996, 2012, Kellogg, 1996; Leijten, van Waes, Schriver, & Hayes, 2014), and discussing the modeling gaps yet to be addressed. Literature-based evidence suggests that there is a need for reconceptualizing the multi-role translating/text production/formulation process in these models, and for explicating the roles of the search for content, reviewing and monitoring processes. Based on analyzing a set of think-aloud protocol data, the author proposes a model in which the written text production process is composed of seven components, and provides detailed descriptions of the strategies each component includes. The paper ends with discussing how this model may contribute to developing the writing ability construct and to building more valid measures for assessing writers’ cognition.

Experts’ reasoning about the replication crisis: Apt epistemic performance and actor-oriented transfer

ABSTRACT Background: The modern world is rife with complex challenges that require citizens to weigh multiple, conflicting claims and competing methods for discerning truth from falsehood. Such evaluations depend highly upon prior knowledge. Therefore, the goal of epistemic education is the cultivation of apt epistemic performance: successfully achieving valuable epistemic aims (e.g., evaluating conflicting claims, discerning truth from falsehood) through competent use of ideals and reliable processes across a range of contexts. However, educators cannot prepare people for every contentious claim they will encounter over their lifetime. Therefore, we investigated if and how apt epistemic performance developed in one discipline could be adapted or transferred for use in another discipline. Methods: We analyzed think-aloud protocol data gathered as experts from psychology, other social sciences (i.e., near transfer), and natural sciences (i.e., far transfer) grappled with a complex problem: psychology’s replication crisis. Findings: An actor-oriented approach best captured how experts outside of psychology were able to near transfer or adapt epistemic performance; far transfer was more difficult. Contribution: Our findings suggest epistemic education for an informed citizenry should prioritize teaching disciplinary norms and practices as well as their scope and limitations, along with self-awareness of when transfer is and is not appropriate.

AN ANALYSIS EFL STUDENTSANALYSIS EFL STUDENTS TRANSLATION OF PREPOSITION IN, ON AND AT A THINK ALOUD- PROTOCOL STUDY TRANSLATION OF PREPOSITION IN, ON AND AT A THINK ALOUD- PROTOCOL STUDY

Translating prepositions in, on and at is not an easy job for EFL students since the meanings relate to locating object, time, and degree. With regard to this point, this study aims at identifying strategies, methods, and procedures used by EFL students to translate preposition in, on and at. Both quantitative and qualitative methods are applied in this study. Think-Aloud Protocol data collecting technique carries out to identify student’s strategies, methods, and procedures in translating prepositions in, on and at. To do this, the data of student’s transcriptions and students’ translations obtained from three students were analyzed. The findings indicate that students use more than one strategies, methods and procedures.

Investigating the effects of client imagery on the ideation process of graphic design

Prior work has examined how images supplied by design clients influence the output of graphic design ideation, however, little is known about the effect upon the ideation process itself. This article reports on an empirical study conducted with graphic design students engaged in design ideation. Findings are based on think-aloud protocol data, as well as participants' subjective ratings of their ideation process using factors based on the principle of design flow. Results show some statistically significant effects on the design process of the participants, particularly increases in their reported anxiety and sense of involvement in the task. In addition, some correlations are identified between design output resulting from exposure to client provided images, and the process of designing those outputs.

Effects of eye movement modeling examples on adaptive expertise in medical image diagnosis

Research indicates that expert performance is domain specific and hardly transfers to novel tasks or domains. However, due to technological changes in dynamic work settings, experts sometimes need to adapt and transfer their skills to new task affordances. The present mixed method study investigates whether eye movement modeling examples (EMME) can promote adaptive expertise in medical image diagnosis. Performance, eye tracking, and think-aloud protocol data were obtained from nine medical experts and fourteen medical students. Participants interpreted dynamic visualizations before (baseline) and after (retention, transfer) viewing an expert model's eye movements. Findings indicate that studying eye movement modeling examples had positive effects on performance, task-relevant fixations, and the use of cognitive and metacognitive comprehension strategies. Effects were stronger for the retention than for the transfer task. Medical experts benefitted more from the modeling examples than did medical students. Directions for future research and implications for related domains are discussed.

Domain-specificity of self-regulated learning processing in science and history

As computer-based learning environments grow in prominence, so do the demands placed upon students to learn with these tools. Empirical research has shown that students who are effective at self-regulating their learning are more likely to acquire deep conceptual understanding while using these environments. However, there is a noticeable lack of research into the degree to which self-regulated learning (SRL) is domain-specific. Investigating this theoretical question about domain-specificity results in related questions about how to best capture and model SRL. To address these concerns, we randomly assigned college students to either a science or history digital library, and used think-aloud protocol (TAP) data to examine the degree to which SRL processing predicted knowledge gains, above and beyond the effects of prior knowledge. We examined multiple methods of aggregating SRL TAP data into analysis variables, to determine which would be the most predictive of learning gains, and then tested these findings using a sample from a second study. In addition, we tested whether the frequency of SRL processing differed by academic domain. We found that data-driven aggregation methods were the most effective at predicting learning gains, and that there were both intriguing similarities in SRL processing across domains (e.g., the importance of corroborating sources) as well as differences (e.g., the predictive validity of self-questioning). Our findings have implications for how to capture and model SRL processing, as well as how to foster SRL among those students who do not yet enact it effectively on their own.

Research on Interpretation Process of Bridging Reference in English Discourse with Think Aloud Protocol

Bridging reference (BR) refers to the relationship between a linguistic expression and its intended referent not explicitly mentioned in previous context. It has been studied for more than three decades with various approaches. However, limitation exists in those approaches except the relevance theory (RT) which suggests that each affecting factor is necessary for the cognitive effects and processing effort required to reach an acceptable overall interpretation. BR interpretation process has seldom been explored and only some questionnaires were adopted in earlier studies. This thesis tentatively approaches BR interpretation process by adopting the think aloud protocol (TAP). In addition, retrospective interview is used to complement TAP. Results from this research indicate that contextual assumptions with cause-effect relation are more accessible in BR interpretation. Contrary to findings from previous researches, contextual assumptions with contrast relation are not considered in this experiment. Encyclopedic information is not always used to test the acceptability of bridging referents. Subjects make choices in order of accessibility of contextual assumptions even though the referent is inconsistent with one’s general knowledge. It supports RT in that one stops inferring once a cognitive effect is achieved. Sometimes one’s activated general knowledge plays a decisive role in BR interpretation. Moreover, encyclopedic information, temporal sequence and syntactic position are taken into consideration in some cases. The constraint of discourse connectives on BR interpretation is proved again in this experiment. It also proves that backward inference is the only comprehension strategy employed by the subjects. The discussion of BR is within the framework of relevance theory. The exploration of BR within RT again verifies the explanatory power of this theory. Moreover, many factors affecting BR interpretation have been studied, but how does each of them work on interpreting BR remains unclear to us. The present study will not only focus on crucial factors affecting BR interpretation but also include other important factors in the analysis. The results from this experiment confirm the validity of TAP in examining BR on the one hand, and suggest alternative research method for later empirical studies of BR on the other hand. It reveals English majors’ cognition process of interpreting BR in English discourse in particular, and their cognition model in second language acquisition (SLA) in general. Analyzing TAP data within RT again validates the great explanatory power of this theory.

The role of reading strategies in integrated L2 writing tasks

Integrated second-language writing tasks elicit writing performances that involve other abilities such as reading or listening. Thus, understanding the role of these other abilities is necessary for interpreting performance on such tasks. This study used an inductive analysis of think-aloud protocol data and interviews to uncover the reading strategies of 12 non-native English writers who completed an integrated reading-writing task. Strategy frequency was considered overall, during composing, and across writers. Word-level strategies were most common along with global and mining strategies. Higher scoring writers used more mining and global strategies, while lower scoring writers showed similar frequency in overall strategy use but differences in choice of strategy. These results suggest that reading plays a role in the process and performance of integrated writing tasks, an important consideration when using such tasks for learning or assessment.

항공관제 상황인식에서 전문가와 초보자의 시선추적 및 프로토콜 분석

Analyses of eye tracking and think-aloud protocol data were performed to examine novice-expert differences in perceptual and cognitive aspects of air traffic controllers' situation awareness. In Experiment 1, three groups of field air traffic controllers (experts, intermediates, novices) were asked to perceive situations that were manipulated by situation complexity. In Experiment 2, protocol analysis for previous situation awareness tasks was performed to extract different task models and strategy models as a function of expertise. Then delayed-recall task and interviews about air control plans for the recalled situations were also executed. Results showed that expert controllers concentrate only on several critical features and have their own strategies to reduce mental workloads.

Effects of Expertise and Situation Complexity on Visual Attention and Action Planning for Air Traffic Control

Analyses of eye tracking and think-aloud protocol data were performed to examine novice-expert differences in perceptual and cognitive aspects of air traffic controllers' situation awareness. In Experiment 1, three groups of field air traffic controllers (experts, intermediates, novices) were asked to perceive situations that were manipulated by complexity. In Experiment 2, protocol analysis of controllers' situation awareness was performed to extract cognitive strategies as a function of expertise. Then delayed-recall task and interviews on air traffic control plans for the recalled situations were also executed. Results showed that expert controllers concentrate only on several critical features and have their own strategies to reduce mental workloads.

A STUDY OF THE ROLE OF AWARENESS IN FOREIGN LANGUAGE BEHAVIOR

This study is a quantitative and qualitative investigation of the effects of awareness, or the lack thereof, on 32 adult second or foreign language (L2) learners' subsequent intake and written production of targeted Spanish morphological forms. Think-aloud protocol data, gathered while learners completed a problem-solving task (a crossword puzzle) and postexposure assessment tasks (a multiple-choice recognition task and a written production task), were used to measure awareness or the lack thereof, and morphological learning was assessed by learners' performances on the two postexposure tasks. From a theoretical perspective, no dissociation between awareness and further processing of targeted forms was found in this study, the results of which are compatible with the claim that awareness plays a crucial role in subsequent processing of L2 data (e.g., Robinson, 1995; Schmidt, 1990, 1993, 1994, 1995). From a methodological perspective, the data collection procedure clearly underscores the need for studies that investigate the roles of attention and awareness in second language acquisition (SLA) to gather as much data as possible from different sources that reveal participants' internal processes. By attempting to ascertain what learners really attend to or are aware of, or both, while exposed to or interacting with L2 data, such information can also address the methodological issue of how representative learners' performances in experimental groups really are in studies conducted under an attentional framework in SLA.

Metaphors of Reasoning as Problem-Solving Tools

This article examines two models of scientific reasoning (dogmatic and sophisticated falsification) as metaphors for the problem-solving activities of a specific work role. Think-aloud protocol data were used to identify problem-solving methods of two electronics experts performing process diagnosis in a very large scale integration (VLSI) fabrication environment. The expert with more experience used sophisticated falsification whereas the expert with less experience used both dogmatic falsification and sophisticated falsification. Computational models (production systems) were constructed and compared with performance of the experts on a sample of diagnostic problems. A neural network was also constructed to examine the development of the two reasoning models. The developmental study suggests that sophisticated falsifications may be a naturally occurring metaphor for diagnosis under the assumption of expectation-based learning and environments of complexity and change.