Talk Aloud Research Articles

Diagnosing and alleviating the impact of performance pressure on mathematical problem solving

High-pressure academic testing situations can lead people to perform below their actual ability levels by co-opting working memory (WM) resources needed for the task at hand (Beilock, 2008). In the current work we examine how performance pressure impacts WM and design an intervention to alleviate pressure's negative impact. Specifically, we explore the hypothesis that high-pressure situations trigger distracting thoughts and worries that rely heavily on verbal WM. Individuals performed verbally based and spatially based mathematics problems in a low-pressure or high-pressure testing situation. Results demonstrated that performance on problems that rely heavily on verbal WM resources was less accurate under high-pressure than under low-pressure tests. Performance on spatially based problems that do not rely heavily on verbal WM was not affected by pressure. Moreover, the more people reported worrying during test performance, the worse they performed on the verbally based (but not spatially based) maths problems. Asking some individuals to focus on the problem steps by talking aloud helped to keep pressure-induced worries at bay and eliminated pressure's negative impact on performance.

Narrative comprehension in Alzheimer's disease: Assessing inferences and memory operations with a think-aloud procedure.

To further understanding of text comprehension abilities in Alzheimer's disease (AD), a think-aloud protocol was used to examine the role of inferencing and the memory operations used to produce inferences. Twenty participants with AD and 20 cognitively healthy older adults (OA) read narratives, pausing to talk aloud after each sentence. A verbal protocol analysis developed by Trabasso and Magliano (1996a) was used to code participants' utterances into inferential and noninferential clause types; inferential statements were then coded to identify the memory operation used in their generation. Compared with OA controls, the AD participants showed poorer story comprehension, d = 2.0, produced fewer inferences, d = .67, and were less skilled at providing explanations of story events, d = 1.27, and in using prior text information to explain outcomes, d = .90. The AD group also appeared to rely more on the activation of world knowledge, d = .58, which contributed to less effective inferences and produced more incoherent noninferential statements, d = 1.05. Poorer text comprehension for the AD group was associated with poorer verbal memory abilities, r's > .55, and poorer use of prior text events when producing explanatory inferences, r = .42. The findings suggest that the memory difficulties of the AD group appear to be an important cognitive factor interfering with their ability to integrate story events through the use of inferences and to create a global coherence to support text comprehension.

Interviews within experimental frameworks: How to make sense of sense-making in virtual worlds

As virtual worlds become increasingly utilized for purposes of entertainment, information and retail, how people understand, think, feel, act and make decisions about them likewise become important research considerations. This essay reports on the methodology and methods used to study these sense-making processes in relatively inexperienced people as they engage with virtual worlds. In order to understand the sense-making of virtual worlds, a method to record the interpretive process, as well as physical actions, was required. In order to understand the sense-making processes involved in new experiences, an amount of control was required over the nature of those experiences. With these requirements, a hybrid study was designed by deconstructing the conceptualization of "the experiment" and utilizing both quantitative and qualitative methods. The resulting study involved the following: a within-subjects experimental design served as the framework for the study, while in-depth qualitative interviews were employed alongside surveys and audio and video recording as the data collection methods. Data collection occurred while participants were engaging with the media products, via talk aloud protocols, and afterwards when they were asked to recall and compare these situations in open-ended questionnaires and interviews structured using Dervin's Sense-Making Methodology. Having completed the study using this mixed method(ology) approach, I discuss the effectiveness of this approach, and where the approach requires more work.

Development and Performance Usability Testing of a Theory-Based, Computerized, Tailored Intervention

Technology offers innovative and promising methods of delivering health messages to provide knowledge and potentially facilitate improved health behaviors. Theory was foundational to the development of a new intervention using a tailored Web site and a handheld computer. A performance usability study was conducted to determine if women could use this newly developed intervention delivered via a Web site and pocket computer accurately and in a timely manner in real-world settings. A convenience sample of 15 women between 35 and 55 years of age were observed as they used "Complete a Task" and "Talk Aloud" performance usability methods. Results identified several problems affecting the ability of participants to use the Web site and handheld computer.

Link maps and map meetings: Scaffolding student learning

With student numbers decreasing and traditional teaching methods having been found inefficient, it is widely accepted that alternative teaching methods need to be explored in tertiary physics education. In 2006 a different teaching environment was offered to 244 first year students with little or no prior formal instruction in physics. Students were invited to attend additional enrichment classes 1 h a week called map meetings. The focus of these classes was a different type of visual presentation of physics material called link maps. Link maps explicitly show the key concepts covered in lectures and how these interrelate to help novices establish their physics schemata. In each map meeting the link map for the different topic was interactively discussed by the researcher before the students worked on problems in groups using the link map. The class ended with the researcher going through one problem, talking aloud about how to logically attack it. The results were promising. Each week about 20% of the class voluntarily attended map meetings whereas 22% reported that they did not attend due to timetable clashes. Two questionnaires revealed that students thought the classes were helpful for gaining an overview of physics and for developing their problem solving abilities. In the final examination the 32 students who had attended at least eight out of ten map meetings achieved, on average, 9 points out of 90 better in the examination $(p=0.004)$ than a comparison group $(N=40)$ with similar academic background which had not attended map meetings. The results of this study suggest that map meetings are a valuable learning environment for physics novices. Further investigations are currently being undertaken.

The Effects of Common Names and Common FR Responses on the Emergence of Stimulus Equivalence Classes

The objective of the present set of studies was compare two procedures for facilitating the emergence of equivalence classes in college students. In Experiment 1, which used a within-group design, participants in one condition were first taught to give class-consistent names to the stimuli which would subsequently be conditionally related. In the second condition participants were first taught to emit common fixed ratio responses in the presence of stimuli which would subsequently be conditionally related. After this training, participants completed conditional discrimination training followed by equivalence testing. Results suggested that the name and FR condition were equally effective in facilitating the emergence of equivalence classes. Experiment 2, which used a between-group design, was similar to Experiment 1 except that participants were asked to “talk aloud” during conditional discrimination training and the stimulus equivalence test. Experiment 2 also employed a control group, who received no common response training. In general, the findings from both experiments suggest that names are not necessary for the formation of equivalence classes, and training common nonverbal responses may be equally effective in promoting class formation.

User‐defined relevance criteria in web searching

PurposeThe purpose of this paper is to specify user‐defined relevance criteria by which people select hyperlinks and pages in web searching.Design/methodology/approachA quantitative and qualitative analysis was undertaken of talking aloud data from nine web searches conducted about self‐generated topics.FindingsAltogether 18 different criteria for selecting hyperlinks and web pages were found. The selection is constituted, by two, intertwined processes: the relevance judgment of hyperlinks, and web pages by user‐defined criteria, and decision‐making concerning the acceptance or rejection of hyperlinks and web pages. The study focuses on the former process. Of the individual criteria, specificity, topicality, familiarity, and variety were used most frequently in relevance judgments. The study shows that despite the high number of individual criteria used in the judgments, a few criteria such as specificity and topicality tend to dominate. Searchers were less critical in the judgment of hyperlinks than deciding whether the activated web pages should be consulted in more detail.Research limitations/implicationsThe study is exploratory, drawing on a relatively low number of case searches.Originality/valueThe paper gives a detailed picture of the criteria used in the relevance judgments of hyperlinks and web pages. The study also discusses the specific nature of criteria used in web searching, as compared to those used in traditional online searching environments.

Methods for Evaluating Online, Resource-based Learning Environments for Teachers

Online delivery methods offer much promise for anytime, anywhere adult learning. As a method of outreach, and to provide support for just-in-time learning, teacher educators are increasingly deciding to design Web sites that are online, resource-based learning environments for teachers and preservice teachers. Automated evaluation tools and data collection methods can help such Web site designers develop and use online, resource-based learning environments to meet their goals of providing high quality learning opportunities for teachers. This article presents a three-pronged approach for evaluating issues of pedagogical design and user-centered functionality in online, resource-based Web sites. Checklist, Web site traffic analysis, and talk aloud protocols art each described and then illustrated through their application to a site aimed at educating teachers about technology integration. We discuss the relevance for developing effective learning environments when teacher educators apply this evaluation approach to resource-based learning environments.

The Effects of Celeration Lines on Visual Data Analysis

Previous visual analysis research reported that the overall agreement between visual analysis and statistical analysis was poor. In response, some researchers suggested the use of celeration lines to improve the accuracy and reliability of visual analysis. However, subsequent research reported little or no improvement in accuracy with such lines. The present study presented 5 board-certified behavior analysts with a series of behavioral graphs. The participants were asked to answer questions similar to those posed in previous studies but were also asked to talk aloud as they viewed each graph. Results indicate that the participants made accurate decisions for only 72% of the graphs and that celeration lines did not improve overall accuracy. The verbal protocol analysis suggests that participants were as likely to attend to trend when celeration lines were absent as they were when they were present, with the most differences attributable to varying participant competencies and not graph (i.e., celeration line) characteristics.

Putting awareness into practice: practical steps for conducting usability tests

PurposeThe purpose of this paper is to describes the methods and findings of two research projects undertaken to explore user behaviour and usability issues relating to the use of Web‐based resources by people with disabilities.Design/methodology/approachWhilst the aims of the research differed, the methods adopted were similar – including observation, talk aloud protocols, questionnaires and focus groups.FindingsThe findings from both studies provided evidence of the problems faced by disabled users when using Web‐based resources. It provided evidence of the types of features users liked and disliked, how they overcame navigational problems and what types of features enhanced their experience, all of which can be fed back into recommendations for the design of electronic resources.Practical implicationsThe richness of the data collected for both studies confirmed the importance of involving users in accessibility and usability assessments. Analysis of both the findings and the methods for both studies also enabled a checklist to be developed, outlining issues that should be taken into consideration when planning a usability test.Originality/valueDocument analysis of past and current usability studies revealed that suggested methods are often directed more towards expert usability testing, rather than managing the user testing in‐house. Although it may be preferable to engage an expert, in reality practical guidance is also needed. The checklist aims to address gaps in usability advice identified, by providing practical guidance to enable providers and developers of web‐based resources to conduct their own effective usability testing.

Methods for inclusion: Employing think aloud protocols in software usability studies with individuals who are deaf

Usability is an important step in the software and product design cycle. There are a number of methodologies such as talk aloud protocol, and cognitive walkthrough that can be employed in usability evaluations. However, many of these methods are not designed to include users with disabilities. Legislation and good design practice should provide incentives for researchers in this field to consider more inclusive methodologies. We carried out two studies to explore the viability of collecting gestural protocols from sign language users who are deaf using the think aloud protocol (TAP) method. Results of our studies support the viability of gestural TAP as a usability evaluation method and provide additional evidence that the cognitive systems used to produce successful verbal protocols in people who are hearing seem to work similarly in people who speak with gestures. The challenges for adapting the TAP method for gestural language relate to how the data was collected and not to the data or its analysis.

CURRENT APPROACHES TO RESEARCHING SECOND LANGUAGE LEARNER PROCESSES

Language learning is a complex set of processes that largely take place in the learner's head. The extent to which learners consciously focus on specific aspects of language, the degree to which they notice particular features of language, and how this is done has been the object of considerable debate in different theoretical approaches to second language acquisition. For researchers in second language acquisition, one dilemma is how to find out what learners notice, and how, if at all, they incorporate this into their developing linguistic knowledge. Here, I discuss three approaches to researching learner cognitive processes that can be used to identify the knowledge that learners have about their second language, and obtain some insights into the cognitive processes of learners. These approaches have the potential to contribute to our understanding of how learners learn a second language, and, therefore, how this task may be facilitated. The first approach attempts to tap directly into the learner's thought through the use of think-aloud protocols, whereas the second involves having learners engage with activities that encourage them to talk aloud, thus providing insights into their thought processes. The third approach uses planning effects on task performance to investigate how learners monitor their language.

Practical Issues in Using a Card Sort in a Study of Nonsupport and Family Caregiving

The authors successfully used the card sort data collection technique with 17 female family caregivers in a large ethnographic study of non-support. In this article, they describe the practical issues they addressed. Initially, they developed strategies to construct meaningful statements that reflected key themes and were manageable in an interview. Later, to address challenges for participants, they developed approaches to assist women in moving beyond their own experience, dealing with test anxiety, and anticipating an emotional response. To facilitate effective data collection, they made detailed arrangements in advance, collected "talk aloud" data that captured women's decisions, and maintained accurate records. The women felt validated in their caregiving roles, as the card statements reflected their experience and rich data was elicited. The card sort exercise contributed information about variability in the meaning of similar interactions for different women and a beginning understanding of the criteria women used to make decisions.

Writing in AAC

Writing in AAC

Reduced communication between frontal and temporal lobes during talking in schizophrenia

Background: Communication between the frontal lobes, where speech and verbal thoughts are generated, and the temporal lobes, where they are perceived, may occur through the action of a corollary discharge. Its dysfunction may underlie failure to recognize inner speech as self-generated and account for auditory hallucinations in schizophrenia.Methods: Electroencephalogram was recorded from 10 healthy adults and 12 patients with schizophrenia (DSM-IV) in two conditions: talking aloud and listening to their own played-back speech. Event-related electroencephalogram coherence to acoustic stimuli presented during both conditions was calculated between frontal and temporal pairs, for delta, theta, alpha, beta, and gamma frequency bands.Results: Talking produced greater coherence than listening between frontal-temporal regions in all frequency bands; however, in the lower frequencies (delta and theta), there were significant interactions of group and condition. This finding revealed that patients failed to show an increase in coherence during talking, especially over the speech production and speech reception areas of the left hemisphere, and especially in patients prone to hallucinate.Conclusions: Reduced fronto-temporal functional connectivity may contribute to the misattribution of inner thoughts to external voices in schizophrenia.

Cortical responsiveness during talking and listening in schizophrenia: an event-related brain potential study

Background: Failures to recognize inner speech as self-generated may underlie positive symptoms of schizophrenia-like auditory hallucinations. This could result from a faulty comparison in auditory cortex between speech-related corollary discharge and reafferent discharges from thinking or speaking, with misattribution of internal thoughts to external sources. Although compelling, failures to monitor covert speech (thoughts) are not as amenable to investigation as failures to monitor overt speech (talking). Methods: Effects of talking on auditory cortex responsiveness were assessed in 10 healthy adults and 12 patients with schizophrenia (DSM-IV) using N1 event-related potentials (ERPs) to acoustic and visual probes during talking aloud, listening to one’s speech played back, and silent baseline. Trials contaminated by muscle artifact while talking were excluded. Results: Talking and listening affected N1 to acoustic but not to visual probes, reflecting modality specificity of effects. Patterns of responses to acoustic probes differed between control subjects and patients. N1 to acoustic probes was reduced during talking compared with baseline in control subjects, but not in patients. Listening reduced N1 equivalently in both groups. Conclusions: Although the failure of N1 to be reduced during talking was not related to current hallucinations in patients, it may be related to the potential to hallucinate.

Effects of controlled breathing, mental activity and mental stress with or without verbalization on heart rate variability

OBJECTIVESTo assess whether talking or reading (silently or aloud) could affect heart rate variability (HRV) and to what extent these changes require a simultaneous recording of respiratory activity to be correctly interpreted.BACKGROUNDSympathetic predominance in the power spectrum obtained from short- and long-term HRV recordings predicts a poor prognosis in a number of cardiac diseases. Heart rate variability is often recorded without measuring respiration; slow breaths might artefactually increase low frequency power in RR interval (RR) and falsely mimic sympathetic activation.METHODSIn 12 healthy volunteers we evaluated the effect of free talking and reading, silently and aloud, on respiration, RR and blood pressure (BP). We also compared spontaneous breathing to controlled breathing and mental arithmetic, silent or aloud. The power in the so called low- (LF) and high-frequency (HF) bands in RR and BP was obtained from autoregressive power spectrum analysis.RESULTSCompared with spontaneous breathing, reading silently increased the speed of breathing (p < 0.05), decreased mean RR and RR variability and increased BP. Reading aloud, free talking and mental arithmetic aloud shifted the respiratory frequency into the LF band, thus increasing LF% and decreasing HF% to a similar degree in both RR and respiration, with decrease in mean RR but with minor differences in crude RR variability.CONCLUSIONSSimple mental and verbal activities markedly affect HRV through changes in respiratory frequency. This possibility should be taken into account when analyzing HRV without simultaneous acquisition and analysis of respiration.

Illuminating Mental Representations Through Speech and Gesture

Can the gestures people produce when describing algebra word problems offer insight into their mental representations of the problems? Twenty adults were asked to describe six word problems about constant change, and then to talk aloud as they solved the problems. Two problems depicted continuous change, two depicted discrete change, and two depicted change that could be construed as either continuous or discrete. Participants' verbal and gestured descriptions of the problems often incorporated information about manner of change. However, the information conveyed in gesture was not always the same as the information conveyed in speech. Participants' problem representations, as expressed in speech and gesture, were systematically related to their problem solutions. When gesture reinforced the representation expressed in the spoken description, participants were very likely to solve the problem using a strategy compatible with that representation—much more likely than when gesture did not reinforce the spoken description. The results indicate that gesture and speech together provide a better index of mental representation than speech alone.

Stimulus Equivalence and The Blocking Effect

The purpose of this experiment was to examine the blocking effect within the context of a stimulus equivalence paradigm. Ten subjects were provided with a prior history of matching unitary sample stimuli to unitary comparisons. Next, the same stimuli appeared as elements of sample stimulus compounds with additional, redundant elements. During the equivalence test, the elements of each compound were separated, and the extent to which the elements had entered separately into equivalence relations with the other stimuli was examined. Five subjects demonstrated the formation of three three-member equivalence classes and showed a strong blocking effect. The other five subjects did not demonstrate the formation of three-member classes, nor did they show a strong blocking effect. The strength of the relations between the elements of each stimulus compound was shown to be consistent with the degree to which the blocking effect was observed. Subjects’ percentage reinforcement estimates were also found to correspond to their performance during the equivalence test. The overt verbal behavior emitted by four subjects who were required to talk aloud during the experiment did not correlate with their nonverbal performance.

The subgoal learning model: Creating better examples so that students can solve novel problems.

Learners have great difficulty solving problems requiring changes to solutions demonstrated in examples. However, if the solution procedures learners form are organized by subgoals, then they are more successful. Subgoal learning is hypothesized to be aided by cues in example solutions that indicate that certain steps go together. These cues may induce a learner to attempt to self-explain the purpose of the steps, resulting in the formation of a subgoal. Across 4 experiments it was found that a label for a group of steps in examples helped participants form subgoals as assessed by measures such as problem-solving performance and talk aloud protocols. Abstract labels were more likely than superficial labels to lead participants to form subgoals with fewer ties to surface features. Subgoals guide problem solving by helping learners focus on the steps to modify in novel problems that involve the same subgoals but require new steps to achieve them.