Talk It Out: Concurrent versus Retrospective Think-Aloud for Complex Verbal Assessment Tasks

This paper reports an explorative experimental comparison of (i) an experience-sampling method called feedback collection and (ii) the think-aloud methods with respect to their usefulness in studies on program comprehension. Think-aloud methods are widely used in studies of cognitive processes, including program comprehension. Alternatively, as in the feedback-collection method (FCM), cognitive processes can be traced by collecting written feedback from the subjects at regular intervals. We compare FCM with concurrent think-aloud (CTA) and retrospective think-aloud (RTA) regarding type and usefulness of the collected information, costs related to analysis of the collected information and effects of the data collection methods on the subjects' performance. FCM allowed us to identify a greater number of comprehension problems that prevented progress or caused significant delay (FCM: 30 problems; CTA: 5; RTA: 15). It was less precise in identifying strategies for comprehension than CTA (92% correctness for FCM; 100% for CTA). FCM was less expensive in analysis (transcription and coding) than the other two methods (FCM: 0.7 h of analysis per protocol; CTA: 31 h; RTA: 7.9 h). The results indicate that all three methods of data collection were intrusive and affected the performance of the subjects with respect to time and correctness (small to medium effect size). This research confirms that FCM can be used beneficially in studies that trace the cognitive processes involved in, and identify problems related to, the comprehension of software applications. On the basis of our experience, we recommend that FCM be used in studies that have a large number of subjects and as a complement to other methods for tracing cognitive processes, such as user log files. We recommend a design with two groups (verbalisation and silent control) and a pretest task to be used in studies with FCM or CTA that focus on performances.

Eye-tracking technology, as an objective-recording method of user’s eye movement data, has widely been applied in human–computer interface usability test. The quantitative analysis of eye movement data is good at comparing different UI designs; however, it can hardly probe the influential factors of usability issues from a UCD perspective. This research aimed at improving the validity of usability test, and tried to fill in the gap of merely applying eye movement data in usability test. In this paper, we compared the characteristics and differences of concurrent and retrospective think-aloud on UI usability evaluation, which were combined with eye-tracking system. We found that concurrent think-aloud would interfere users during the tests, thus increased their mental stress and cognitive workload, which suits assessing simple-operation and low-cognitive-workload tasks; the retrospective think-aloud enables user to operate more naturally, thus is proper for more complex task chain testing. Moreover, the retrospective think-aloud is more efficient for analyzing the cognitive factors of usability issues in-depth.

Compromised Memory Function in Schizophrenia and Temporal Lobe Epilepsy

A Comparison of Left-Handed and Right-Handed Subjects on Verbal and Non-Verbal Dichotic Listening Tasks

In a study of verbal temporal ordering, patients with Parkinson's disease (PD) were impaired in making judgements of the relative recency of serially-presented words but were normal in word recognition; by contrast, patients with Alzheimer's disease (AD) were impaired on both verbal tasks (Sagar et al., 1985, 1988b). The present study used a nonverbal analogue of the verbal temporal ordering task to examine whether the deficit in recency discrimination of the PD patients transcended material-specific bounds or was specific to verbal material. The nonverbal task was administered to the patients with PD, patients with AD and normal control subjects studied with the verbal task. The results showed that the retention functions for the recognition of verbal and nonverbal material differed qualitatively from each other and this difference was present in all three subject groups. For verbal and nonverbal recency discrimination and verbal content recognition, performance declined with increasing retention intervals; for nonverbal content recognition, however, performance was essentially unchanging across retention intervals. As on the verbal test, recency discrimination on the nonverbal test was impaired in both patient groups. In contrast to the results with the verbal task, both groups were impaired on content recognition. The patterns of impairment, however, differed between the two patient groups in a parallel way to those of the verbal task: performance of the AD group was impaired at the longest retention intervals but not at the shortest intervals, whereas the performance of the PD group was impaired at the shortest but not at the longest intervals. This deficit in nonverbal short-term recognition memory was present even in PD patients who scored within the normal range on a quantitative test of dementia. These results show that deficits in recency discrimination and short-term memory processing in PD extend to nonverbal as well as verbal material and imply bilateral frontal lobe dysfunction in the genesis of these deficits.

Simulations and games bring the possibility to research complex processes of managerial decision-making. However, this modern field requires adequate methodological procedures. Many authors recommend the use of a combination of concurrent think-aloud (CTA) or retrospective think-aloud (RTA) with eye-tracking to investigate cognitive processes such as decision-making. Nevertheless, previous studies have little or no consideration of the possible differential impact of both think-aloud methods on data provided by eye-tracking. Therefore, the main aim of this study is to compare and assess if and how these methods differ in terms of their impact on eye-tracking. The experiment was conducted for this purpose. Participants were 14 managers who played a specific simulation game with CTA use and 17 managers who played the same game with RTA use. The results empirically prove that CTA significantly distorts data provided by eye-tracking, whereas data gathered when RTA is used, provide independent pieces of evidence about the participants’ behavior. These findings suggest that RTA is more suitable for combined use with eye-tracking for the purpose of the research of decision-making in the game environment.

BackgroundTourists are at risk of experiencing health problems during their travel. However, even though tour guides have the potential to become travel health promoters, their participation has not been optimal.ObjectiveThis study aims to develop a comprehensive travel health education model to help tour guides improve health information delivery to tourists.MethodsThis is an exploratory sequential mixed methods research. The first phase consisted of a qualitative study with an informed grounded theory design. In-depth interviews were carried out with tour guides from all language divisions and policymakers of the Indonesian Tour Guide Association Bali Branch or Himpunan Pramuwisata Indonesia Daerah Bali (HPI Bali). The interview guidelines were developed based on the theory of planned behavior and identity theory. Qualitative data were analyzed thematically. In the interim phase, a travel health education model and questionnaire were developed based on the qualitative findings. The initial model and its instruments were finetuned after consultation with travel medicine and health promotion experts. Furthermore, the validity and reliability of the questionnaire were tested on 30 tour guides. The second phase consisted of a quantitative study with a randomized pretest-posttest control group design. A total of 76 tour guides in the intervention group received comprehensive travel health education, while 76 in the control group received no specific intervention. Outcome variables (ie, attitudes, subjective norms, perceived behavioral control, actual behavioral control, role identity, and behavioral intention) were measured at baseline (T0), after the online training (T1), before information sharing via WhatsApp (T2), a month after the start of the WhatsApp intervention (T3), and at the end of the WhatsApp intervention (T4). The mean difference of each outcome variable before and after the intervention will be compared between the intervention and control groups. Thereafter, the quantitative and qualitative findings will be integrated into a joint display.ResultsThe qualitative phase was conducted through in-depth interviews with 21 informants who included tour guides and policymakers from HPI Bali from May to June 2021. The education model, educational materials, and questionnaire were developed based on the qualitative findings and consultation with experts. The education model consists of online training and information sharing through WhatsApp and was trialed with tour guides from November 2021 to February 2022. As of April 2022, this study is in the quantitative data analysis stage.ConclusionsA travel health education model was developed based on qualitative findings and consultation with experts. The model was tested with tour guides, and a series of self-administered questionnaires were completed. This study is in the quantitative data analysis stage and will continue by integrating qualitative and quantitative findings into a joint display.Trial RegistrationClinicalTrials.gov NCT04961983; https://clinicaltrials.gov/ct2/show/NCT04961983

We compare the effectiveness of two types of verbal protocol, concurrent think aloud vs. retrospective sense making, for evaluating the usability of a clinical decision support tool. Thirty-five medical and nursing students participated in a usability experiment. Participants were asked to complete seven tasks using the system under evaluation. Eighteen students were allocated to the concurrent think aloud group and the remainder followed the retrospective protocol. The retrospective protocol was significantly more sensitive than the concurrent protocol in recording unique usability problems related to users' cognitive behaviour. These problems concerned the interpretation and comprehension of statistical output, search results and system messages. These findings can be explained by the retrospective protocol's greater ability to detect compound usability problems, capturing the cognitive dimensions of users' interactions with the interface in greater depth. Evaluations of clinical decision support systems should take an evidence-based approach to selecting protocols.

This paper presents a case study aiming to investigate which variant of the Think-Aloud Protocol (i.e., the Concurrent Think-Aloud and the Retrospective Think-Aloud) better integrates with the Cognitive Walkthrough with Users. To this end we performed a case study that involved twelve users and one usability evaluator. Usability problems uncovered by each method were evaluated to help us understand the strengths and weaknesses of the studied usability testing methods. The results suggest that 1) the Cognitive Walkthrough with Users integrates equally well with both the Think-Aloud Protocol variants; 2) the Retrospective Think-Aloud find more usability problems and 3) the Concurrent Think-Aloud is slightly faster to perform and was more cost effective. However, this is only one case study, and further research is needed to verify if the results are actually statistically significant.

The traditional EFL classroom is no longer enough to provide sufficient possibilities for EFL learners' verbal engagement, which has become one of the most challenging for EFL teachers inside classroom settings. This study aims to extend EFL children's oral interactions outside classroom settings by providing their parents with bilingual explainer videos and video materials designed for this purpose. The study uses interviews, pretests, and posttests as tools for data collection, and participants of the control and experimental groups were taught the same content via the traditional method. However, the supplementary videos (explainer videos & video materials) were only used to be shown to the parents of the experimental group's participants. Data analysis revealed that the experimental group’s participants whose parents employed the supplementary videos outperformed much better than the control group's participants whose parents just depended on the students’ books. Hence, bilingual explainer videos and material videos are useful tools for helping EFL parents extend speaking practices with their children at home. The children's spoken performance in several areas of the language has improved dramatically. The participants progress from nodding their heads to delivering two complex phrases. Thus, it suggests designing supplementary videos to serve as a guide for EFL parents to engage in verbal interactive tasks with their children at home.

Social perception skills, such as understanding the mind and emotions of others, affect children's communication abilities in real-life situations. In addition to autism spectrum disorder (ASD), there is increasing knowledge that children with specific language impairment (SLI) also demonstrate difficulties in their social perception abilities. To compare the performance of children with SLI, ASD and typical development (TD) in social perception tasks measuring Theory of Mind (ToM) and emotion recognition. In addition, to evaluate the association between social perception tasks and language tests measuring word-finding abilities, knowledge of grammatical morphology and verbal working memory. Children with SLI (n = 18), ASD (n = 14) and TD (n = 25) completed two NEPSY-II subtests measuring social perception abilities: (1) Affect Recognition and (2) ToM (includes Verbal and non-verbal Contextual tasks). In addition, children's word-finding abilities were measured with the TWF-2, grammatical morphology by using the Grammatical Closure subtest of ITPA, and verbal working memory by using subtests of Sentence Repetition or Word List Interference (chosen according the child's age) of the NEPSY-II. Children with ASD scored significantly lower than children with SLI or TD on the NEPSY-II Affect Recognition subtest. Both SLI and ASD groups scored significantly lower than TD children on Verbal tasks of the ToM subtest of NEPSY-II. However, there were no significant group differences on non-verbal Contextual tasks of the ToM subtest of the NEPSY-II. Verbal tasks of the ToM subtest were correlated with the Grammatical Closure subtest and TWF-2 in children with SLI. In children with ASD correlation between TWF-2 and ToM: Verbal tasks was moderate, almost achieving statistical significance, but no other correlations were found. Both SLI and ASD groups showed difficulties in tasks measuring verbal ToM but differences were not found in tasks measuring non-verbal Contextual ToM. The association between Verbal ToM tasks and language tests was stronger in children with SLI than in children with ASD. There is a need for further studies in order to understand interaction between different areas of language and cognitive development.

Music in the major mode is often associated with happy feelings, which could enhance task performance, compared with that in the minor mode, which is associated more with sadness. Male and female participants (N = 48) completed written verbal and spatial reasoning tests while a piece of music in F major by Handel was being played, and again when the same piece was digitally manipulated to create a version in the minor mode. The confounding variable of using two different compositions was thus avoided. Results showed that the music in the major mode was rated more emotionally positive by both sexes than was the minor mode version (p ≤ .001). Performance by females on verbal tasks was significantly enhanced with major mode music, compared with the minor (p = .018), but there were no such findings for other combinations of sex and task. Also with major mode music only, there were trends for females to score higher than males on verbal tasks, and for males to score the highest on spatial tasks. Reasons for the research findings are suggested.

Purpose — The purpose of this chapter is to provide knowledge and insights gained into Canadian expatriates' perspectives of living and working in Indonesia and to enhance the reader's understanding of the lived experience of being immersed in another culture.Methodology — A descriptive phenomenological qualitative method was employed, which uncovered paradoxes that were lived by Canadian expatriate managers.Findings — The following paradoxes are presented using participants' words and lived experiences of the authors: powerful-powerless, understanding-not understanding, being supported-not supported, belonging-not belonging and freedom-restriction.Research limitations — While the qualitative findings may not be 'generalisable' according to a quantitative viewpoint, they are certainly applicable to others' experiences as demonstrated by the authors' description of their own experiences.Practical implications — Enhanced understanding of this lived experience will spark improved expatriation strategies and lead directly to significant improvement in the overall performance of international organisations because an effective expatriate will be more capable of facilitating the two-way sharing and blending of the local and exogenous knowledge, therefore fulfilling a key role in knowledge transfer.Social implications –– The importance of being open to others' lived experiences and supporting others who are immersed in another culture has universal appeal from a social perspective.Originality/value of chapter — This is a unique research chapter describing research findings and joining participants' words and experiences with the authors' experiences. As well it is hoped that readers will relate to the paradoxes.

Effects of alpha and gamma transcranial alternating current stimulation (tACS) on verbal creativity and intelligence test performance

Досліджено роль і значення поняття «закріплення доказів» у структурі процесу доказування. Підтримано позицію вчених про віднесення поняття «закріплення доказів» до етапу їх збирання. Із посиланням на комп’ютеризацію даних у суспільстві, що постійно збільшується, запропоновано доповнити ч. 2 ст. 84 КПК України додатковим видом процесуального джерела доказів – «комп’ютерні дані». Також запропоновано відносити до змісту конструкції «закріплення доказів» спеціальні кримінальні процесуальні правила й порядок перетворення комп’ютерних даних у писемну форму, придатну для безпосереднього сприйняття людиною та збереження задля подальшого використання в доказуванні.