Think-aloud Study Research Articles

WoTs-TDGV: Thing Description Generator and Validator

The breakneck evolution of the Web of Things (WoT) has led to a burgeoning demand for reliable tools that facilitate the automatic generation and validation of Thing Descriptions (TD). However, emerging tools that generate or validate TD automatically have suffered from abounding limitations and constraints. In this paper, we propose and implement an automatic web-based Thing Description Generator and Validator (TDGV) application. TDGV automates and simplifies TDs generation while ensuring the compliance with the WoT standards. The application generates more accurate thing description results. In order to evaluate the usability of the TDGV, a Concurrent Think-aloud usability study was conducted with a sample of 150 developers. The study revealed positive feedback regarding the user interface and the TDGV usability. The study results highlight areas for enhancement, such as TDGV navigation. TDGV is a convenient and robust Thing Description Generator and Validator web application. It provides a valuable tool for users in the WoT domain, enabling them to automatically and easily generate and validate TDs.

Gender Differences in Sexual Violence Victimization Experiences and Validity of Victimization Reports: A Think-Aloud Study

ABSTRACT This study compared the qualitative nature of women and men’s sexual violence (SV) victimization, the types of experiences captured and missed on the Sexual Experiences Survey-Short Form Victimization (SES-SFV) across genders, and common interpretations of the SES-SFV items. Fifty-four university students (31 women, 21 cis men, 2 trans men) who had recent unwanted (but not necessarily nonconsensual) sexual experiences thought out loud while privately completing the SES-SFV. They also typed descriptions of experiences reported on SES-SFV items or similar experiences when nothing was reported on an item. Results indicated that women’s victimization was more frequent and severe than cis men’s, except when men were victimized by men. Although verbal coercion was common across genders, event descriptions indicated that women’s verbal coercion experiences were more often harsh and part of a partner’s ongoing SV or coercive control. The findings suggest that quantitative measurement can mask important gender differences in victimization and (based on analysis of false positives and negatives) may underestimate rape and attempted rape experiences, especially women’s. Findings suggested that responding to the SES-SFV was not traumatic or distressing. However, participants sometimes expressed confusion about the items and interpreted them in unintended ways.

Which street characteristics support cycling for transport among vulnerable groups in traffic: A think-aloud study in virtual reality

Although several health benefits are associated with cycling, this mode of active transport remains underrepresented. Studies have shown that individuals' preferred mode of transport is influenced by the environment in which they travel. While macro-environmental factors shape active transport, micro-environmental aspects offer adaptable solutions. In this paper, we used novel and promising Virtual Reality (VR) technology to investigate which street characteristics regarding safety and attractiveness are important for promoting cycling for transport in three vulnerable groups in traffic, i.e., adolescents, adults with a low SES, and older adults. Participants cycled through two virtual streets based on existing streets in Ghent, Belgium. The think-aloud method was used to collect qualitative data during cycling and afterward participants were asked to fill in a questionnaire about the VR experience, self-reported cycling behavior, and sociodemographic factors. Transcribed files were analyzed using content analysis, revealing 14 crucial street characteristics. Separate cycling paths, clear markings and traffic signs, even surfaces, calm streets, spacious layouts, and aesthetically pleasing environments were found to be important by all target groups. Adolescents and older adults emphasized slower traffic, visibility, wider cycle paths, high curbs, and sudden unexpected actions. Secure bicycle parking was crucial for adolescents, while older adults valued green environments and had mixed opinions on traffic-calming structures. This study underscores the significance of micro-scale street features in promoting active transport among vulnerable groups. Additionally, it demonstrates the potential of VR for inclusive urban design and citizen-based science.

Reliability and validity of the Turkish version of the medication practical barriers to adherence questionnaire in patients with chronic diseases.

The aim of this study was to evaluate the validity and reliability of the Turkish version of the Medication Practical Barriers to Adherence Questionnaire (MPRAQ-TR). This is a validation study conducted between August 2022 and March 2023 in the ambulatory services of a secondary care private hospital located in Istanbul, Türkiye, among patients (18years of age and older) with chronic diseases. After the Turkish translation and cultural adaptation of the MPRAQ, and a pilot think-aloud study, the following psychometric properties were assessed: internal consistency by calculating the Cronbach's alpha coefficient, 2-week test-retest reliability, convergent validity by calculating Spearman's rank correlation between the MPRAQ-TR and the Turkish version of Medication Adherence Report Scale (MARS), and predictive validity by evaluating the association between the MPRAQ-TR score and nonadherence to medications. Among the 380 patients (response rate = 89.6%), 72.1% were nonadherent to their medications. The intraclass correlation coefficient was 0.99 for MPRAQ-TR scores (95% CI, 0.98-0.99; p < 0.001). The Cronbach's alpha of the MPRAQ-TR was 0.853. There was a moderate negative correlation between the scores on the MARS and the MPRAQ-TR (Spearman's rho = - 0.525; p < 0.01), supporting the convergent validity of the MPRAQ-TR. In the univariate analysis, the total score of MPRAQ-TR was associated with increased odds of nonadherence to medications (p < 0.01). MPRAQ-TR shows good psychometric properties and can be used to evaluate the practical adherence barriers of patients with chronic diseases.

Misleading information in crises: exploring content-specific indicators on Twitter from a user perspective

ABSTRACT Recent crises like the COVID-19 pandemic provoked an increasing appearance of misleading information, emphasising the need for effective user-centered countermeasures as an important field in HCI research. This work investigates how content-specific user-centered indicators can contribute to an informed approach to misleading information. In a threefold study, we conducted an in-depth content analysis of 2382 German tweets on Twitter (now X) to identify topical (e.g. 5G), formal (e.g. links), and rhetorical (e.g. sarcasm) characteristics through manual coding, followed by a qualitative online survey to evaluate which indicators users already use autonomously to assess a tweet's credibility. Subsequently, in a think-aloud study participants qualitatively evaluated the identified indicators in terms of perceived comprehensibility and usefulness. While a number of indicators were found to be particularly comprehensible and useful (e.g. claim for absolute truth and rhetorical questions), our findings reveal limitations of indicator-based interventions, particularly for people with entrenched conspiracy theory views. We derive four implications for digitally supporting users in dealing with misleading information, especially during crises.

Visual Exploration and Analysis of Simulation and Testing Data in Motor Engineering.

End-of-line tests and defect detection are vital for ensuring the reliability of electric motors. However, automated defect detection methods (e.g., data-driven approaches) face challenges due to the limited availability of real data from failed motors. Simulated data, though beneficial, lacks the complexity of real motors, impacting the performance of these methods when applied to actual observations. To tackle this challenge, we introduce a visual analysis tool designed to facilitate the analysis of measured and simulated data, presented in the form of time series data. This tool helps identify domain-invariant features and evaluate simulation data accuracy, assisting in selecting training data for reliable automated defect detection in real-world scenarios. The main contribution of this work is a design proposal based on visual design principles, specifically tailored to address the unique requirements of electric motor professionals. The visual design is validated by findings from a think-aloud study with specialized engineers.

Are the content and usability of a new direct observation tool adequate for assessing competency in delivering person-centred care: a think-aloud study with patients and healthcare professionals in Sweden

ObjectiveTo evaluate the content and usability of a new direct observation tool for assessing competency in delivering person-centred care based on the Gothenburg Centre for Person-Centred Care (gPCC) framework.DesignThis is...

Immersive Virtual Colonography Viewer for Colon Growths Diagnosis: Design and Think-Aloud Study

Desktop-based virtual colonoscopy is a proven and accurate process for identifying colon abnormalities. However, it is time-consuming. Faster, immersive interfaces for virtual colonoscopy are still incipient and need to be better understood. This article introduces a novel design that leverages VR paradigm components to enhance the efficiency and effectiveness of immersive analysis. Our approach contributes a novel tool highlighting unseen areas within the colon via eye-tracking, a flexible navigation approach, and a distinct interface for displaying scans blended with the reconstructed colon surface. The path to evaluating and validating such a tool for clinical settings is arduous. This article contributes a formative evaluation using think-aloud sessions with radiology experts and students. Questions related to colon coverage, diagnostic accuracy, and time to complete are analyzed with different user profiles. Although not aimed at quantitatively measuring performance, the experiment provides lessons learned to guide other researchers in the field.

Foundations for Enabling People to Recognise Misinformation in Social Media News based on Retracted Science

For many people, social media is an important way to consume news on important topics like health. Unfortunately, some influential health news is misinformation because it is based on retracted scientific work. Ours is the first work to explore how people can understand this form of misinformation and how an augmented social media interface can enable them to make use of information about retraction. We report a between-subjects think-aloud study with 44 participants, where the experimental group used our augmented interface. Our results indicate that this helped them consider retraction when judging the credibility of news. Our key contributions are foundational insights for tackling the problem, revealing the interplay between people's understanding of scientific retraction, their prior beliefs about a topic and the way they use a social media interface that provides access to retraction information.

P261: Metaverse gamified aesthetic education – a think-aloud usability pilot study

Abstract Introduction Ten years on from the Keogh Review of cosmetic surgery in the UK, improvements in training and education are yet to be achieved. Gamified learning and metaverse technology have been posited as transformational media for learning. This study evaluates the usability of a metaverse gamified platform for aesthetic education. Methods An aesthetic education platform for administration of Botulinum Toxin was developed in Unity Game Engine. Qualitative semi-structured interviews were conducted using Nielsen’s (1994) usability testing of end-users, combining task-based assessment with free exploration of the platform. Articulated thoughts were transcribed, whilst the platform was screen recorded. Verbal and observational data were thematically analysed and coded, whilst barriers and facilitators were rated on a modified Likert scale. Results A total of eight medical students and junior doctors were included in the study. Six higher themes were developed including aesthetic interface, familiarity, coherence, interactivity, ease of use and utility. Coherence and interactivity were identified as the most potent drivers of learning, whilst familiarity and aesthetic interface were the biggest potential barriers. Conclusion A metaverse gamified educational platform in aesthetic plastic surgery is potentially usable and useful for doctors and medical students as a learning tool. Important facilitators include focus on in-game coherence and interactivity. Further work is needed to develop educational experiences further within metaverse settings in aesthetic plastic surgery to enhance education and promote patient safety.

They Don’t Read Very Well: A Study of the Reading Comprehension Skills of English Majors at Two Midwestern Universities

Abstract: This paper analyzes the results from a think-aloud reading study designed to test the reading comprehension skills of 85 English majors from two regional Kansas universities. From January to April of 2015, subjects participated in a recorded, twenty-minute reading session in which they were asked to read the first seven paragraphs of Charles Dickens’ Bleak House out loud to a facilitator and then translate each sentence into plain English. Before subjects started the reading tests, they were given access to online resources and dictionaries and advised that they could also use their own cell phones as a resource. The facilitators also assured the subjects that were free to go at their own pace and did not have to finish reading all seven paragraphs by the end of the exam. As part of the study, each subject filled out a survey collecting personal data (class rank, G.P.A., etc.) and took a national literacy exam (the Degrees of Reading Power Test 10A ). After the 85 taped reading tests were completed, the results were transcribed and coded.

The process of clinical decision-making in chronic wound care: A scenario-based think-aloud study

AimsTo undertake a comprehensive investigation into both the process of information acquisition and the clinical decision-making process utilized by primary care nurses in the course of treating chronic wounds. DesignScenario-based think-aloud method, enriched by the integration of information processing theory. The study was conducted within the framework of home care nursing organizations situated in Flanders, the Flemish speaking part of Belgium. A cohort of primary care nurses (n = 10), each possessing a minimum of one year of nursing experience, was recruited through the collaboration of three home care nursing organizations. MethodsTwo real-life clinical practice scenarios were employed for the interviews, with the researcher adopting the roles of either the patient or another clinician to enhance the realism of the think-aloud process. Each think-aloud session was promptly succeeded by a subsequent follow-up interview. The Consolidated criteria for Reporting Qualitative research checklist was followed to guarantee a consistent and complete report of the study. ResultsAmidst noticeable variations, a discernible pattern surfaced, delineating three sequential concepts: 1. gathering overarching information, 2. collecting and documenting wound-specific data, and 3. interpreting information to formulate wound treatment strategies. These concepts encompassed collaborative discussions with stakeholders, while the refinement of wound treatment strategies was interwoven within both concepts 2 and 3. ConclusionsEvident variations were identified in chronic wound care clinical decision-making, regardless of educational background or experience. These insights hold the potential to inform the development of clinical decision support systems for chronic wound management and provide guidance to clinicians in their decision-making endeavours.

Usability, Appeal, and Relevance of Music and Wellbeing Training for Carers of People with Dementia: A Think-Aloud Study.

Personalized music listening can be highly effective in supporting people experiencing changes in mood and behavior due to Alzheimer's disease and other conditions causing dementia. However, a lack of staff education can be a barrier to effective use of music in residential aged care. This study investigates the usability, appeal and relevance of an online training course designed to help care staff develop personalized music programs for people living with dementia in their care. A mixed methods approach was taken in which 13 participants took part in a think-aloud session while using the training. Qualitative data from the think-aloud session and interviews were triangulated with results of a survey evaluation of the training. Themes relating to Engagement, Usability and Appeal, Pedagogical Design and Content, and Transfer, Impact and Barriers to Uptake were discussed. Results indicated that participants regarded the virtual environment and pedagogical design as of high quality, although some minor issues in navigability were identified. Participants also found the training to be highly relevant to their roles as caregivers and reported ways they would incorporate key concepts into care practices. However, ongoing barriers to implementation of such training were identified including systemic issues that contribute to a culture of focus on physical care. Consistency of navigation and clear instructions for interacting with content are important in training of time-poor care staff. Linking training to industry standards can provide important motivation for implementation of training, although systemic barriers can impede real-world change.

Iraqi Graduate Students’ Analysis of the Graduate Record Examination (GRE) Verbal Reasoning Content: A Think-Aloud Study

This study describes difficulties Iraqi graduate students encounter while answering Verbal Reasoning Measure (VRM) questions of the Graduate Record Examination (GRE). Participants thought aloud while answering questions from three VRM subsections: Reading Comprehension (RC), Text Completion (TC), and Sentence Equivalence (SE). Data were collected via audio recordings, from seven Iraqi graduate students, and transcribed orthographically, coded, and analyzed. The analysis yielded several themes: Difficulties related to the structure of the question and/or the meaning of its answers, the content is pertinent to USA culture and history, SE is the most difficult part followed by TC, and RC is time consuming. These findings are consistent with previous research results that the structure of the GRE VRM questions and/or the answers are difficult to understand, SE items were much harder than TC and RC items due to their unfamiliar content and vague meanings, and RC is time consuming for international graduate students. Pedagogical implications were identified and recommendations for ETS and students were suggested.

A think-aloud study: Exploring the effects of digital vs. print comics on reading efficiency and comprehension

A think-aloud study: Exploring the effects of digital vs. print comics on reading efficiency and comprehension

How do adolescents experience a newly developed Online Single Session Sleep Intervention? A Think-Aloud Study.

Sleep problems are common in adolescents and have detrimental impacts on physical and mental health and daily functioning. Evidence-based treatment like cognitive behaviour therapy for insomnia (CBT-I) is often hard to access, and adolescents may not engage in and adhere to longer, clinician-delivered interventions. Brief, self-guided, and accessible sleep interventions are needed. To explore the user experience of a prototype online self-help single session sleep intervention developed for adolescents. Eleven participants aged 17-19years (8 females, 3 males) took part in online retrospective think-aloud interviews. Participants first completed the prototype intervention independently and were then shown the intervention page by page and asked to verbalise their thoughts and experiences. Transcripts were analyzed thematically. Participants found the intervention helpful. Four themes were generated - 'Educative: Learning, but more fun', 'Effortless: Quicker and Easier', 'Personalization: Power of Choice', and 'Positivity: Just Good Vibes'. The theme 'Educative: Learning, but more fun' encompassed two sub-themes 'Opportunity to Learn' and 'Aesthetics and Learning'. These themes reflected participants' views that the intervention was educative, personalised, solution-oriented and easy to use, but could incorporate more graphics and visuals to aid in learning and could be made more effortless and positive through modifications to its design. Findings convey the importance of ensuring educative well-designed content, personalization, a positive tone, and ease of use while designing interventions targeting adolescents's sleep and mental health. They also indicate areas for further developing the intervention.

Face Validity of Four Preference-Weighted Quality-of-Life Measures in Residential Aged Care: A Think-Aloud Study.

There is an increased use of preference-weighted quality-of-life measures in residential aged care to guide resource allocation decisions or for quality-of-care assessments. However, little is known about their face validity (i.e., how understandable, appropriate and relevant the measures are 'on their face' when respondents complete them). The aim of this study was to assess the face validity of four preference-weighted measures (i.e., EQ-5D-5L, EQ-HWB, ASCOT, QOL-ACC) in older people living in residential aged care. Qualitative cognitive think-aloud interviews were conducted using both concurrent and retrospective think-aloud techniques. To reduce burden, each resident completed two measures, with the four measures randomised across participants. Audio recordings were transcribed and framework analysis was used for data analysis, based on an existing framework derived from the Tourangeau four-stage response model. In total, 24 interviews were conducted with residents living across three residential aged care facilities in Melbourne, Australia. Response issues were identified across all four measures, often related to comprehension and difficulty selecting a response level due to double-barrelled and ambiguous items that have different meanings in the residential aged care context. We also identified issues related to understanding instructions, non-adherence to the recall period, and noted positive responding that requires attention when interpreting the data. Our findings provide further evidence on the appropriateness of existing measures, indicating numerous response issues that require further research to guide the selection process for research and practice.

Content validity of the EQ-5D-5L with skin irritation and self-confidence bolt-ons in patients with atopic dermatitis: a qualitative think-aloud study.

Two bolt-on dimensions (skin irritation, self-confidence) have been developed for the EQ-5D-5L to improve its content validity and responsiveness in psoriasis. However, the two bolt-ons are not strictly psoriasis-specific and are potentially relevant in other skin conditions. This study aims to explore the content validity of the EQ-5D-5L with two bolt-ons in patients with atopic dermatitis (AD). In 2021-2022, qualitative, semi-structured interviews were conducted with 20 adult AD patients at a university dermatology clinic in Hungary. We aimed for a heterogeneous sample in terms of age, gender, education and disease severity. Patients completed the EQ-5D-5L with two bolt-ons using a think-aloud protocol. Probing questions were posed to investigate item relevance, potential conceptual overlaps, missing concepts and the appropriateness of the recall period. Interview transcripts were subjected to thematic analysis. The EQ-5D-5L with the two bolt-ons covered the most important aspects of health-related quality of life in AD patients. Most patients found both the skin irritation and self-confidence bolt-ons relevant. Fifteen potential missing concepts were identified, but only two (social relationships, judgement by others) were identified by more than one patient. A smaller conceptual overlap was found between the skin irritation and pain/discomfort dimensions in 7 patients (35%). Half the patients expressed a preference for a recall period of 1week rather than of 'today'. The EQ-5D-5L with skin irritation and self-confidence bolt-ons showed good relevance, comprehensiveness and comprehensibility in patients with AD. However, in terms of comprehensiveness, social relationships and judgement by others (stigma) may be missing from the questionnaire.

Humorous hate speech on social media: A mixed-methods investigation of users’ perceptions and processing of hateful memes

Humor that denigrates social groups can be just as harmful as hate speech. Despite research indicating the prevalence of humorous hate speech, how audiences perceive and process the combination of humor (e.g. irony as a humor cue) and hate speech (e.g. dehumanization as a hate cue) remains unclear. Using a sequential mixed-methods approach combining a qualitative think-aloud study (Study 1, N = 41) with an experiment involving implicit measurements of response times (Study 2, N = 65), it was examined how individuals perceive memes that contain both humor and hate cues. While think-aloud interviews indicated that processing humorous hate speech may require multiple steps, the relative time spent by participants in Study 2 to rate humorous and non-humorous hate speech as being hostile or not did not entirely support that conclusion. However, findings imply that hostile views may become more commonplace when hate speech is masked by humor.

Content validity of the post-traumatic growth inventory: a think-aloud study on capturing recovery from addiction

Background There is emerging evidence that post-traumatic growth is a relevant concept in addiction recovery. However, existing measures, such as the post-traumatic growth inventory, were designed to measure post-traumatic growth in general trauma populations. It is unclear if the post-traumatic growth inventory is suitable for individuals in recovery from substance-related addiction. The current study aimed to qualitatively assess the content validity of the post-traumatic growth inventory for individuals in addiction recovery. Method The think-aloud method was applied, and semi-structured interviews utilized, to collect data from 20 individuals in recovery from unhealthy alcohol and/or drug use. Reflexive thematic analysis was used to analyze data in relation to item interpretation and experience of post-traumatic growth. Findings Qualitative assessment supported three out of five subscales on the post-traumatic growth inventory (new possibilities, personal strength and appreciation of life). Modifications were suggested on some items from the relating to others domain and responses to the spiritual change domain possibly reflected a cultural influence. There was an identified need to include items that account for positive changes in health behavior. Conclusions The current study highlights domains of post-traumatic growth for those in addiction recovery and how the post-traumatic growth inventory may understate some forms of positive life change. Future research should focus on validating a revised version of the post-traumatic growth inventory for this population.