Retrospective Verbal Protocols Research Articles

ELF Density: Extending English as a Lingua Franca Research to Monological ELF Texts and Speeches

Abstract The focus to date on interactive encounters has resulted in there being little research into monological texts and speeches produced in English as a lingua franca (ELF) contexts. They are, however, the very substance of what interpreters and translators increasingly deal with today. The quality of these language professionals’ performance depends to a large extent on the input they receive. From a translation and interpreting studies (TIS) perspective, understanding what makes an ELF text or speech special and different seems paramount. In this paper, we present an innovative approach to exploring what is in an ELF text. We introduce the concepts of ‘ELF density’ and ‘ELF dense spots’ to capture and visualize types and tokens of ELF features as well as clusters they form at a local and more global level across stretches of text or speech and discuss a number of methodological challenges in determining ELF density. Based on a preliminary retrospective verbal protocol and interview analysis following 26 professional interpreters’ renditions, we aim to demonstrate the concepts’ added value for TIS, but also for non-TIS related applied linguistic tasks geared towards ELF.

Investigating cognitive processes in different item formats in reading tests through eye-tracking and verbal protocols

This study has investigated the differences in cognitive processes that test-takers undergo while answering reading comprehension questions in multiple-choice and open-ended short answer formats. For this purpose, data were collected from a group of undergraduate students in an English medium university through eye-tracking technology, immediate retrospective verbal protocols, and short semi-structured interviews. The results showed that the participants used careful reading skills more and comprehended the text more thoroughly in the open-ended format. However, in the MC format, they read less carefully and used more test-taking strategies. These findings contribute to the ongoing discussion on how item format can alter the cognitive processes in a reading comprehension test and confirm the effectiveness of eye-tracking in unveiling cognitive processes in combination with qualitative methods. This study has implications for reading test development.

How Teachers Adapt Their Cognitive Strategies When Using Entrustment Scales.

Rater cognition is a field of study focused on individual cognitive processes used by medical teachers when completing assessments. Much has been written about the role of rater cognition in the use of traditional assessment scales. Entrustment scales (ES) are becoming the tool of choice for workplace-based assessments. It is not known how moving to an entrustment framework may cause teachers to adapt their cognitive rating strategies. This study aimed to explore this gap by asking teachers to describe their thinking when making rating decisions using a validated ES. Using purposive sampling, family medicine teachers supervising obstetrical care were invited to participate in cognitive interviews. Teachers were interviewed between December 2018 and March 2019 using retrospective verbal protocol analysis. They were asked to describe their experiences of rating residents in the last 6 months using new ES. Constructivist grounded theory guided data collection and analysis. Interviews were recorded, transcribed, and analyzed iteratively. A constant comparative approach was used to code and analyze the data until consensus was reached regarding emerging themes. There was variability in how teachers used the ES. Faculty describe several ways in which they ultimately navigated the tool to say what they wanted to say. Four key themes emerged: (1) teachers interpreted the anchors differently based on their cognitive framework, (2) teachers differed in how they were able to cognitively shift away from traditional rating scales, (3) teachers struggled to limit assessments to a report on observed behavior, and (4) teachers contextualized their ratings. Variability in teachers' interpretation of learner performance persists in entrustment frameworks. Rater's individual cognitive strategies and how they observe, process, and integrate their thoughts into assessments form part of a rich picture of learner progress. These insights can be harnessed to contribute to decisions regarding readiness for unsupervised practice.

Improvising on Emotion Terms: Students’ Strategies, Emotional Communication, and Aesthetic Value

Studies in musical improvisation show that musicians and even children are able to communicate intended emotions to listeners at will. To understand emotional expressivity in music as an art form, communicative success needs to be related to improvisers’ thought processes and listeners’ aesthetic judgments. In the present study, we used retrospective verbal protocols to address college music students’ strategies in improvisations based on emotion terms. We also subjected their improvisations to expert ratings in terms of heard emotional content and aesthetic value. A qualitative analysis showed that improvisers used both generative strategies (expressible in intramusical terms) and imaginative, extramusical strategies when approaching the improvisation tasks. The clarity of emotional communication was found to be high overall, and linear mixed-effects models showed that it was supported by generative approaches. However, perceived aesthetic value was unrelated to such emotional clarity. Instead, aesthetic value was associated with emotional complexity, here defined as the heard presence of “nonintended” emotions. The results point toward a view according to which the expressive content of improvisation gets specified and personalized during the very act of improvisation itself. Arguably, musical expressivity in improvisation should not be equated with the error-free communication of previously intended emotional categories.

First-person, video-stimulated recall method for studying musical improvisation strategies

Accessing the strategies implemented during real-time musical creative processes represents a challenge for researchers. Retrospective verbal protocol with subjective aided recall (RVPwSAR), where a first-person video is used as an aid to elicit feedback from a subject, has shown great potential for the study of real-time decision-making in sport, therapy, emergency situations, and consumer behavior. This article reviews research methods studying real-time decision-making and discusses the implications of a previous study using RVPwSAR to observe decision-making strategies during musical improvisation. The objective of this article is twofold: (1) to present the contributions and limitations of the RVPwSAR to arrive at a better understanding of the strategies implemented in real-time during musical improvisation; (2) to discuss the potential of the RVPwSAR for studying the strategies implemented in other real-time musical creative tasks. I argue that research where the RVPwSAR is applied may contribute to a better understanding of the strategies used during the real-time musical process, and, ultimately, contribute to the promotion of creative musical teaching and learning.

Speaking of your own repertoire: an investigation of music performance during practice

AbstractThe purpose of this study was to explore musicians’ approaches to performance during practice and identify the factors that underpinned their approaches. We hypothesised that musicians would be able to recall their focus, knowledge and thoughts of their own repertoire during music performance and that such data would reveal musicians’ cognitive behaviours during the performance. By analysing musicians’ retrospective verbal protocols, we found that musicians used four main reasoning processes –study, static analysis, intuitionandperformer’s analysis –in their approach to music performance. The findings show that musicians utilise multiple cognitive behaviours for music performance. The implications for instrumental music teaching are discussed.

What You might not be Assessing through a Multiple Choice Test Task

Comprehending a text involves constructing a coherent mental representation of it and deep comprehension of a text in its entirety is a critical skill in academic contexts. Interpretations on test takers’ ability to comprehend texts are made on the basis of performance in test tasks but the extent to which test tasks are effective in directing test takers towards reading a text to understand the whole of it is questionable. In the current study, tests based on multiple choice items are investigated in terms of their potential to facilitate or preclude cognitive processes that lead to higher level reading processes necessary for text level macrostructure formation. Participants’ performance in macrostructure formation after completing a multiple choice test and a summarization task were quantitatively and qualitatively analyzed. Task performances were compared and retrospective verbal protocol data were analyzed to categorize the reading processes the participants went through while dealing with both tasks. Analyses showed that participants’ performance in macrostructure formation of the texts they read for multiple choice test completion and summarization task differed significantly and that they were less successful in comprehending the text in its entirety when they were asked to read to answer multiple choice questions that followed the text. The findings provided substantial evidence of the inefficacy of the multiple choice test technique in facilitating test takers’ macrostructure formation and thus pointed at yet another threat to the validity of this test technique.

An Analysis of Students’ Cognitive Bias in Experimental Activities Following a Lab Manual

The purpose of this study is to develop a thinking process model that reveals cognitive bias through analyzing students’ cognitive biases in processing experimental manuals. Twenty-two college students participated in the study. During the “making electromagnets” experimental activity, we collected students’ concurrent verbal protocols, gaze positions, and experimental behaviors. After the experiment, we collected their retrospective verbal protocols and ensured reliability by diversifying the data. The collected data were analyzed inductively using the grounded theory methodology. The results showed that four categories of paradigm (causal conditions, phenomena, interactions, and contextual conditions) and fifteen concepts were derived. Students displayed bias in following the manual instructions due to the influence of causal conditions. When embodying biased representations as workspace entities, biased responses come from the influence of contextual conditions. Therefore, these can be developed in consideration of causal and contextual conditions when developing a manual, thereby reducing cognitive bias among students, and ultimately helping them perform accurate experiments.

Evidence of cognitive effort in simultaneous interpreting: Process versus product data

The present study aims at measuring the level of correspondence between the problems related to increased cognitive effort reported by simultaneous interpreting trainees in their retrospective verbal protocols and problems indicators identified in the target texts. The study triangulates process analysis (retrospective protocols) with product analysis (manual comparison of source and target texts). The corpus of the study consists of about 75 hours of recordings of 240 interpreting outputs and the recordings of 5,005 retrospective comments. The results imply that that increased cognitive load involved in simultaneous interpreting and the resulting cognitive effort experienced by trainee interpreters is not always manifested in the product.

Investigando o Esforço Cognitivo, o conhecimento sobre Tradução e a Satisfação na Tradução LIBRAS-Português

Este trabalho apresenta os resultados de um estudo exploratório-experimental que utilizou rastreamento ocular, registro de ações de teclado e mouse, protocolos verbais retrospectivos livres e questionários prospectivo e de satisfação como ferramentas metodológicas para as suas coletas e análises de dados. A fundamentação teórica apresenta aportes de diferentes áreas, como dos estudos empírico-experimentais com línguas de sinais (EMMOREY; THOMPSON; COLVIN, 2009; WEHRMEYER, 2014) e dos estudos processuais da tradução (JAKOBSEN; JENSEN, 2008; PAVLOVIĆ; JENSEN, 2009; HVELPLUND, 2017, dentre outros), que se apoiam em conceitos e princípios dos estudos da cognição. Os resultados obtidos e as análises e discussões desenvolvidas corroboram a validade e a produtividade do modelo de desenho experimental aplicado para abordar as questões em tela, além de trazerem contribuições relevantes para os ramos dos Estudos da Tradução em que se inserem e potencializarem questões para futuras pesquisas na área.

Gender and Age Differences in Using Indoor Maps for Wayfinding in Real Environments

Users more easily become lost in complex indoor environments than in outdoor environments. Users with diverse backgrounds encounter different self-location, route memorization, and route following problems during wayfinding. This study intends to explore gender and age effects on the use of indoor maps for wayfinding in real environments. We used eye-tracking and retrospective verbal protocol methods to conduct a wayfinding experiment in a newly opened building. Statistical data were collected and three findings were obtained. Finding 1: Males had no significant differences with females in indoor self-location, route reading, and route following. However, males paid less visual attention to the landmark and legend than females during route reading. Finding 2: Age-related differences were significant in indoor wayfinding. Younger adults generally outperformed elderly adults in wayfinding in real indoor environments. Finding 3: Gender and age interactive effects were significant in self-location and route memorization. The mean differences of visual attention on the self-location map reading and route memorization between males and females increased with age.

Mitigating Unwarranted Forecasting Behavior in the Newsvendor Task

We use a combination of eye tracking and retrospective verbal protocol analysis (VPA) to map the decision making process in the newsvendor task. In Study 1, we found that decision makers in the classic repeated newsvendor task setting show a propensity to forecast demand using perceived trends in the past demand when actually forecasting is normatively inappropriate in this context. We explored two interventions to rectify this pattern of behavior. First, by suppressing the temporal dimension (Supp_Temp_Dim), that is, asking the participants to make newsvendor decisions for 20 different products for the same period instead of making the newsvendor decision 20 times for the same product, we observed a decrease in the extent of forecasting behavior. Second, by imposing standing orders (Stdg_Ord), we observed that it increased the forecasting horizon. In Study 2, we tested an intervention that sensitized the subjects to the critical fractile logic (Crit_Frac) and found that it improved financial performance. The forecasting behavior persisted but the orders now fluctuated around a higher mean than before. The improved understanding of the fundamental cognitive processes that lurk behind the observed decisions can translate into improved real world decision making.

Expert Western Classical Music Improvisers’ Strategies

The growing interest in musical improvisation is exemplified by the body of literatures evidencing the positive impacts of improvisation learning on the musical apprentice’s aptitudes and the increasing presence of improvisation in Western classical concert halls and competitions. However, high-level Western classical music improvisers’ thinking processes are not yet thoroughly documented. As a result of this gap, our research addresses the following question: What strategies are implemented in the course of performance by Western classical music improvisers? To answer this question, semistructured interviews were conducted with five internationally recognized Western classical music expert improvisers. Each participant improvised, and immediately afterward, a retrospective verbal protocol with subjective aided recall data collection strategy was used to elicit the improvising musician’s strategies. After transcription, the interviews were coded and analyzed using NVivo 10 software, with a mixed (i.e., combining inductive and deductive coding) category approach. Our data revealed 46 improvisation strategies that were subsequently organized into five categories: preplanning, conceptual, structural, atmospheric and stylistic, and real time. Pedagogical implications arising from these findings are that (a) learners should be guided toward implementing various strategies and (b) the capacity to switch from one strategy to another according to the circumstances should be promoted.

DIRECTIONALITY: TYPES AND FREQUENCY OF REPAIRS IN SIMULTANEOUS INTERPRETATION

The article describes an experimental study of self-corrections in simultaneous in­terpretation (SI) in relation to directionality while making use of previous studies in this field on cognitive processes and strategies in SI. The paper aims to contri­bute to the literature regarding repair strategies the interpreter resorts to during SI, and examine their possible link to directionality between the language pairs Lithuanian-English and Turkish-English by way of an experimental study. The ex­periment was carried out with 8 students each working in two directions, from En­glish (B) into Lithuanian (A) or Turkish (A) and the reverse. The material analysed comprises the types and frequency of self-repairs in SI and the reflections on the retrospective verbal protocols provided by the subjects, prompted by listening to the source texts and their interpretations directly after the interpreting task. The results of the study suggest that the types and frequency of repairs differ greatly when interpreting into A and B. The conclusions of the study present some peda­gogical implications for training future interpreters.

“Communication is a two-way street”

Speech act studies are increasingly likely to use retrospective verbal protocols to record the thoughts of participants who produced targeted speech acts (e.g., Cohen & Olshtain, 1993). However, although communication is always a two-way street, little is known about the recipients’ perceptions of speech acts. In academic communication at universities, it is critical for students to gain awareness of the socio-cultural norms as well as knowledge of appropriate linguistic forms in interacting with instructors. Therefore, gathering perceptual information from instructors, the recipients of many speech acts such as apologies, serves an important role in realizing successful student-instructor communication. Targeting instructors’ perceptions, two forms of an online survey were created via surveygizmo.com, with one including 12 spoken apologies and the other including 12 emailed apologies. An equal number of native (NS) and nonnative English speaking (NNS) students produced these apologies. The 150 instructors who responded to the survey gave significantly higher ratings to apologies made by NS students than to those made by NNS students. An analysis of instructors’ explanations after the ratings showed that both sociopragmatic and pragmalinguistic knowledge (Thomas, 1983) were valued in the successful realization of apologies, with the majority of instructor explanations addressing the sociopragmatic aspects of apology productions. In their comments on highly-rated student apologies, instructors appreciated the fact that students took responsibility in apologizing, offered worthy explanations, and delivered the messages with minimum grammatical mistakes. Poorly rated apology messages did not contain sufficient or valid evidence, inconvenienced the instructors through inappropriate requests, and usually had multiple grammatical mistakes. This study provides useful source of information to be used in university classrooms that can orientate novice learners towards socio-cultural expectations and appropriate lexical markers to be employed in making successful apologies in academic settings.

Applying Aspects of the Expert Performance Approach to Better Understand the Structure of Skill and Mechanisms of Skill Acquisition in Video Games.

Video games are ideal platforms for the study of skill acquisition for a variety of reasons. However, our understanding of the development of skill and the cognitive representations that support skilled performance can be limited by a focus on game scores. We present an alternative approach to the study of skill acquisition in video games based on the tools of the Expert Performance Approach. Our investigation was motivated by a detailed analysis of the behaviors responsible for the superior performance of one of the highest scoring players of the video game Space Fortress (Towne, Boot, & Ericsson, ). This analysis revealed how certain behaviors contributed to his exceptional performance. In this study, we recruited a participant for a similar training regimen, but we collected concurrent and retrospective verbal protocol data throughout training. Protocol analysis revealed insights into strategies, errors, mental representations, and shifting game priorities. We argue that these insights into the developing representations that guided skilled performance could only easily have been derived from the tools of the Expert Performance Approach. We propose that the described approach could be applied to understand performance and skill acquisition in many different video games (and other short- to medium-term skill acquisition paradigms) and help reveal mechanisms of transfer from gameplay to other measures of laboratory and real-world performance.

Arguing with oneself

Argumentation is generally conceived of as a dialogic activity between two or more participants. Nonetheless, it operates also at an intrapersonal level (Rocci 2005), in a soliloquy where protagonist and antagonist of the critical discussion are embodied in the same person. We argue this case by analyzing journalists’ argumentation about linguistic choices in newswriting processes. Empirically, we draw on data generated with progression analysis (Perrin 2003), in particular with cue-based retrospective verbal protocols. The data was produced by the journalists under investigation when they, while watching video recordings of their text production processes, reconstructed and verbalized their decisions (Perrin 2011: 60). In the detail analysis, we focus on one editorial by an experienced journalist ofCorriere del Ticino,the main Italian-language newspaper in Switzerland.

TEXTUAL AND INTER-TEXTUAL ANALYSES OF IRANIAN EFL UNDERGRADUATES’ TYPES OF ENGLISH READING TOWARDS DEVELOPING A CAREFUL READING FRAMEWORK

This study investigated textual and inter-textual reading of a group of Iranian EFL undergraduates’ careful English reading types. In this research, Khalifa and Weir’s (2009) reading framework was used to propose a more inclusive aspect of a careful reading framework and the reading construct for instructional and assessment goals. The participants of this study were B.A. students of English Translation at Shiraz Payame Noor University. To obtain the required data, a questionnaire and a careful reading test along with reading journals, interviews, and retrospective verbal protocols were used. The findings revealed that careful reading at the sentential and textual levels were seen to be practiced frequently by the participants. However, reading purposes and cognitive processes requiring integrating information from different texts, reading critically to establish and evaluate the authors’ position on a particular topic, building links across texts, judging the relatedness of texts, evaluating the writer’s ideas and comparing viewpoints were not seen as prevalent emerged reading patterns. The participants performed differently on tasks measuring different types of careful reading at different levels in a descending order of difficulty. In doing the tasks, although no statistically significant difference was found between the performance of males and females, they performed differently regarding their age and educational level differences. Subsequently, based on the outcomes, in the proposed careful reading framework, some new variables such as educational level, age, documents knowledge, better understanding and careful reading at multiple text level structures were added to Khalifa and Weir’s (2009) reading framework.

An experimental approach to understanding design problem structuring strategies

This study has developed an experimental approach to understanding the effectiveness of problem structuring strategies such as decomposition modes and control strategies. Participants were randomly recruited to individually complete a design problem in a lab, and their design outcomes, including sketches and retrospective verbal protocols, were collected. The effectiveness of problem structuring was represented as the ratio of output to input of the human information processing system: the output was the quality of design outcomes, and the input was participants' mental effort normalised by the complexity of problem structuring. The case study has shown that the participants who applied the explicit decomposition mode and the breadth-first strategy had significantly higher effectiveness scores than those who applied the implicit mode and the depth-first strategy. The results suggest that effective problem structuring strategies should be encouraged in design idea generation and design education.

Becoming a beer expert: Is simple exposure with feedback sufficient to learn beer categories?

Category learning is an important aspect of expertise development which had been little studied in the chemosensory field. The wine literature suggests that through repeated exposure to wines, sensory information is stored by experts as prototypes. The goal of this study was to further explore this issue using beers. We tested the ability of beer consumers to correctly categorize beers from two different categories (top- and bottom-fermented beers) before and after repeated exposure with feedback to beers from these categories. We found that participants learned to identify the category membership of beers to which they have been exposed but were unable to generalize their learning to other beers. A retrospective verbal protocol questionnaire administrated at the end of the experiment indicates that contrary to what was suggested in the wine literature, prototype extraction is probably not the only mechanism implicated in category learning of foods and beverages. Exemplar-similarity and feature-frequency models might provide a better account of the course of learning of the categorization task studied.