Total Error Count Research Articles

Usability Evaluation Ecological Validity: Is More Always Better?

The ecological validity associated with usability testing of health information technologies (HITs) can affect test results and the predictability of real-world performance. It is, therefore, necessary to identify conditions with the greatest effect on validity. We conducted a comparative analysis of two usability testing conditions. We tested a HIT designed for anesthesiologists to detect pain signals and compared two fidelity levels of ecological validity. We measured the difference in the number and type of use errors identified between high and low-fidelity experimental conditions. We identified the same error types in both test conditions, although the number of errors varied as a function of the condition. The difference in total error counts was relatively modest and not consistent across levels of severity. Increasing ecological validity does not invariably increase the ability to detect use errors. Our findings suggest that low-fidelity tests are an efficient way to identify and mitigate usability issues affecting ease of use, effectiveness, and safety. We believe early low-fidelity testing is an efficient but underused way to maximize the value of usability testing.

Impact of on-call shifts on working memory and the role of burnout, sleep, and mental well-being in trainee physicians

ABSTRACT Background Optimal cognitive functions, including working memory (WM), are crucial to enable trainee physicians to perform and excel in their clinical practice. Several risk factors, including on-call shifts, poor mental health, burnout, and sleep problems, can impair clinical practice in trainee physicians, potentially through cognitive impairment; however, these associations have not been fully explored. Objective This study investigated the effect of on-call shifts on WM among trainee physicians and its association with burnout, depression, anxiety, affect, and sleep. Materials and methods This cross-sectional study involved 83 trainee physicians (45% male). We measured demographic and training-related factors including on-call shifts and working hours. We also assessed depressive symptoms (PHQ-9), both state and trait anxiety (STAI total score), burnout (OLBI total score), positive and negative affect scores (PANAS), and sleep disturbances (PSQI total score). WM was evaluated using spatial working memory (SWM) strategy scores that reflected performance and total error counts. Results Trainee physicians with more on-calls per month had significantly worse depressive symptoms, burnout scores, and sleep, as well as more negative affect. While controlling for covariates, being on-call more times per month was significantly associated with worse WM. Worse depressive symptoms and burnout scores were also significantly associated with impaired WM. Conclusion Working more on-call shifts is associated with compromised WM. Trainee physicians who experienced more depressive symptoms and burnout had worse WM.

Exploring the potential of ChatGPT in assessing L2 writing accuracy for research purposes

This study investigates ChatGPT's potential for measuring linguistic accuracy in second language writing for research purposes. We processed 100 L2 essays across five proficiency levels with ChatGPT-4 and manually coded for precision and recall with regard to ChatGPT's identification of errors. Our findings indicate a strong correlation (ρ = 0.97 using one method and .94 using another method) between ChatGPT's error detection and human coding, although this correlation diminishes with lower proficiency levels. While ChatGPT infrequently misidentifies errors, it often underestimates the total error count. The study also highlights ChatGPT's limitations, such as the issue of consistency, and provides guidelines for future research applications.

The Acute Effect Of Mindful High-intensity Interval Training On Inhibitory Control, Attention, And Cognitive Flexibility

PURPOSE: High-intensity interval training (HIIT) has been shown to have acute benefits to cognitive performance, however, the existing literature has exclusively focused on the exercise bouts while ignoring the recovery intervals, which typically consist of passive breaks. This is despite compelling evidence indicating additional cognitive gains resulting from exercise combined with cognitive engagement. Properly designed recovery intervals during HIIT can be an important opportunity to deliver mindful activity, a mental exercise of self-awareness and self-regulation that has acute benefits to attention and executive function. This study investigated whether a novel mindful HIIT intervention has acute facilitating effects on inhibitory control, attention, and cognitive flexibility performance. METHODS: Using a within-subject crossover design, 59 young adults (21 ± 0.9 yrs, 22 males, BMI = 24 ± 3) completed a 30-min bout of sedentary control and mindful HIIT on separate days in counterbalanced order. The mindful HIIT included a 3-min warm-up, 3-min cool-down, and 24-min main activity consisting of three 4-min bouts of body-weight exercise separated by three 4-min bouts of mindful recovery. During the main activity, participants achieved 76% of age-predicted (208-0.7*age) maximum heart rate (HRmax) and ratings of perceived exertion (RPE) = 15. Following each condition, participants completed the stroop, d2, and trial-making tasks to measure their inhibitory control, attention, and cognitive flexibility. RESULTS: No difference in inhibitory control during stroop task or in cognitive flexibility during the trial-making task was observed between mindful HIIT and control conditions (ts ≤ 1.53, ps ≥ .13, ds ≤ 0.20). Participants showed lower omission error count (t = 2.49, p = .02, d = 0.32), total error count (t = 2.34, p = .02, d = 0.30), and total error percent (t = 2.37, p = .02, d = 0.29) during the d2 task following the mindful HIIT compared with control condition. CONCLUSIONS: These findings provide the first empirical evidence that mindful HIIT is a feasible approach to induce acute facilitation of cognitive performance. Further, this acute mindful HIIT effect may be selective for facilitating attention but not inhibitory control and cognitive flexibility.

Fatigue-Recovering Effect of a House Designed With Open Space

Exposure to a natural environment has been reported to be associated with positive effects on mental well-being. However, no report has examined the effects of a house designed with an open space connected to nature on recovery from fatigue. The purpose of this study was to examine the effects of such an open space on recovery from mental fatigue. Placebo-controlled, crossover design. Participants were randomized into open (connected to nature) and closed (not connected to nature) conditions. Sixteen healthy female volunteers were enrolled. After a 30-minute fatigue-inducing mental task, participants moved to an open or closed recovery room for 30 minutes. As fatigue-evaluating mental tasks, participants performed advanced trail making tests for 20 minutes. They were asked to rate their levels of fatigue, relaxation, comfort, and healing on a visual analogue scale from 0 (minimum) to 100 (maximum) to evaluate their subjective mental. They also underwent accelerated plethysmography. After the recovery session, lower total error counts of a cognitive test, greater levels of subjective relaxation, comfort, and healing, and lower levels of waveform index-1 assessed via accelerated plethysmography were observed in participants exposed to the open condition compared with the closed condition. These results provide evidence that the use of a house designed with an open space connected to nature during the recovery session improved cognitive function and subjective mental states. Hence, open space was effective for helping recovery from mental fatigue.

Effects of pellet stove on recovery from mental fatigue

SummaryBackgroundExposure to a warm environment has been reported to be effective for recovery from mental fatigue. However, there have been no reports examining the effects of a pellet stove on recovery from mental fatigue. The purpose of this study was to examine the effects of a pellet stove on recovery from mental fatigue.Material/MethodsIn this placebo-controlled, crossover experiment, 16 healthy volunteers were randomized into the pellet stove and control groups. After a 30-min fatigue-inducing mental task session, participants moved to a recovery room with (pellet stove condition) or without (control condition) a pellet stove to see the image of a pellet stove for 30 min.ResultsAfter the recovery session, the participants exposed to the pellet stove condition showed lower total error counts of a cognitive test, higher levels of subjective healing, comfort, and warmth, and sympathetic nerve activity and higher parasympathetic nerve activity as compared with the control condition.ConclusionsThese results provide evidence that improved cognitive function, subjective mental states, and balance of the autonomic nervous activities result from using a pellet stove during the recovery session. Hence, the pellet stove was effective for the recovery from mental fatigue.

Intelligibility of normal speakers: Error analysis

Errors made by listeners in sentence transcription were analyzed to determine their contributions to intelligibility. One hundred Harvard sentences produced by ten males and ten females were transcribed by ten listeners per talker. Two measures of intelligibility were obtained: a keyword score in which a sentence was correct if, and only if, all five keywords were correct; and a total error count. Analysis of error types revealed that typing/spelling errors accounted for a third of total errors, and phonetic errors (consonant and vowel errors) accounted for another third. The remainder were semantic errors, added or deleted words, or unclassifiable. Further analysis of consonant errors did not reveal any particular type of consonant to be more susceptible to error than others. Male talkers had significantly worse keyword scores than females but did not have significantly greater total error counts indicating more errors on function words for females. The difference between high and low intelligibility speakers (by total error count) was accounted for by increased typing/spelling and consonant errors. Acoustic analysis of incorrectly transcribed words revealed that phonetic errors appeared to originate in the mouths of speakers while errors like word substitutions arise in the ears (or brains) of listeners.

Diagnostic Efficacy of Plasma Urokinase-type Plasminogen Activator and Plasminogen Activator Inhibitor-2 in Differentiation of Hepatocellular Carcinoma from Cirrhosis

We determined the plasma antigen levels of urokinase-type plasminogen activator(u-PA) and plasminogen activator inhibitor 2(PAI-2) in 41 patients with hepatocellular carcinoma and 28 patients with different stages of liver cirrhosis. No significant differences of u-PA and PAI-2 levels were calculated between the two groups of tumor patients (HCC) and liver cirrhosis without tumor (non-HCC). Within both study groups, no significant differences were found in u-PA and PAI-2 levels of the different Child categories. Discriminative functions of both u-PA and PAI-2 (total error count estimates of 43.1% and 43.6%, respectively), were low compared to that (29.0%) of alpha-fetoprotein (AFP). The combinations of AFP and u-PA lowered the total error rate (21.9%) more than that of each marker alone. However, whether plasma u-PA and PAI-2 may be considered as a risk factor further investigation was needed and our findings raise the question as to whether these markers could be considered as useful screening markers for earlier detection of HCC in liver cirrhosis because discriminant functions of u-PA and PAI-2 were not significant. Sensitivities and specificities of u-PA and PAI-2 were also not high enough, resulting in the ranges of total diagnostic efficiency from 43% to 50%, and, from 49% to 63%, respectively, at different cut-off values. No direct relationship was detected between AFP and u-PA, between AFP and PAI-2, and between u-PA and PAI-2.

Effects of intracerebroventricular administration of ethylcholine aziridinium ion on hippocampal cholinergic system, assayed in vivo by a brain dialysis technique, and on spatial memory of rats

AbstractWe attempted to clarify the effect of hippocampal function on the spatial memory of rats by creating a lesion in the hippocampus with ethylcholine aziridinium ion (AF64A). The hippocampus was damaged by injecting AF64A into the lateral ventricles, 3 nmol/3 μl per side. At 1 or 4 weeks after the injection, acetylcholine (ACh) was measured in perfusion samples collected via brain dialysis of the hippocampus of freely moving rats. The release of ACh stimulated by scopolamine was reduced at 1 week but not at 4 weeks post‐injection. A behavioral change on the eight‐arm radial maze test was also observed in the acquisition or retention trials to determine the deficit in spatial memory, whether acquisition or retention. In the acquisition trial, the initial correct response was decreased and the total error count was increased at 1.5 to 2.5 weeks, but not at 3 weeks, after the AF64A injection. However, those indices remained unchanged in the retention trial. These observations suggest that the hippocampus may be required for the acquisition of spatial memory, but that the stored memory, but the stored memory is located outside the hippocampus, the hippocampus is not concerned with the retrieval process, or the retrieval process may be hippocampal but not cholinergic. The hippocampal cholinergic system would be concerned with the acquisition of spatial memory. Thus, AF64A‐treated rats may serve a model for hippocampal cholinergic dysfunction and the recovery from hippocampal cholinergic damage. © 1993 Wiley‐Liss, Inc.