Task Performance Data Research Articles

How Does Software Visualization Contribute to Software Comprehension? A Grounded Theory Approach

Despite their ability to synthesize vast amounts of information, software visualization tools are not widely adopted in the software engineering industry. In an effort to investigate the underlying reasons, we conducted a usability study to investigate the affordances of software visualization techniques for the maintenance of complex software systems. Expert programmers were asked to carry out programming tasks with or without using a software visualization tool while their screens and eye gaze patterns were recorded. Statistical analysis of task performance data showed that participants who used the software visualization tool outperformed the control group in terms of task completion time and accuracy. However, quantitative analysis of performance measures did not reveal in what ways software visualizations contributed to this improvement. In an effort to identify the cognitive strategies that underlie this quantitative performance difference, process models grounded in qualitative analysis of eye-tracking data were constructed. The process models indicated that software visualizations guided the subjects in the experiment group toward following specific software comprehension strategies, which account for the difference observed in task performance data.

Multiple Correspondence Analysis of Fuzzyfied Task Performance and Psycho-Physiological Test Data: Use in Real Car Following Situations

Abstract This article shows the role that Fuzzy Space Windowing (FSW) and Multiple Correspondence Analysis (MCA) may play in the prospect of analyzing two different data sets separately, first and together, afterwards. This methodology is illustrated with a car-following example with 123 individuals. The first data set contains each individual's psychological profile, obtained through tests and questionnaires (23 variables); the second data set contains the signals acquired from real-life driving situations with an instrumented vehicle (6 time variables). The MCA of the first set yields individual typologies, for example, that conscientious drivers declare fewer errors in driving than impulsive and stressed drivers, or that drivers with high cognitive levels have a tendency to declare more violations than other drivers do. The MCA of the second set highlights inter-driver differences (e.g., speed, headway and pedal use). Other factors, such as infrastructure, also have an impact on driving behavior. For example, drivers have a tendency to drive under the speed limit on urban roads and over the speed limit on highways. A third and final MCA is then performed on the most informative variables found with the two previous MCAs.

Demonstration of a Human Task Inventory to Support Rapid Emulation of Human Performance in Aviation Simulations

Simulation testbeds enable researchers in the aviation domain to study a wide variety of NextGen issues and are particularly useful in circumstances where human-in-the-loop testing is impractical or impossible. In such cases, human performance data is required to accurately represent operators within the simulation framework. We are developing the Human Task Inventory (HTI) to address this issue by providing a centralized repository of pilot and controller task performance data. This web-based tool will allow users to efficiently locate, evaluate, and export data. When possible, task data from multiple studies are combined to produce better estimates of response distributions. Users are also able to evaluate data sources and choose which sources to include in the calculations. Our demonstration will include an initial HTI prototype and a description of envisioned functionality.

Error correction maintains post‐error adjustments after one night of total sleep deprivation

Previous behavioral and electrophysiologic evidence indicates that one night of total sleep deprivation (TSD) impairs error monitoring, including error detection, error correction, and posterror adjustments (PEAs). This study examined the hypothesis that error correction, manifesting as an overtly expressed self-generated performance feedback to errors, can effectively prevent TSD-induced impairment in the PEAs. Sixteen healthy right-handed adults (seven women and nine men) aged 19-23 years were instructed to respond to a target arrow flanked by four distracted arrows and to correct their errors immediately after committing errors. Task performance and electroencephalogram (EEG) data were collected after normal sleep (NS) and after one night of TSD in a counterbalanced repeated-measures design. With the demand of error correction, the participants maintained the same level of PEAs in reducing the error rate for trial N + 1 after TSD as after NS. Corrective behavior further affected the PEAs for trial N + 1 in the omission rate and response speed, which decreased and speeded up following corrected errors, particularly after TSD. These results show that error correction effectively maintains posterror reduction in both committed and omitted errors after TSD. A cerebral mechanism might be involved in the effect of error correction as EEG beta (17-24 Hz) activity was increased after erroneous responses compared to after correct responses. The practical application of error correction to increasing work safety, which can be jeopardized by repeated errors, is suggested for workers who are involved in monotonous but attention-demanding monitoring tasks.

Proposed components of an “ideal” indoor noise criteria rating system.

A number of studies have been conducted at the University of Nebraska on correlating human performance and perception to indoor noise criteria systems. Task performance and subjective perception data were gathered from subjects exposed to background noise conditions commonly due to mechanical systems, including some with tonal components and some with time‐varying fluctuations. Results show that perceptions of annoyance and distraction are highly correlated to the character of the noise, but the current indoor noise criteria systems (such as noise criteria and room criteria) do not accurately reflect that relationship. This paper presents a proposal for what an indoor noise criteria rating system should ideally include, to quantify acceptable building mechanical system noise in commercial buildings. [Work supported by the American Society of Heating, Refrigeration and Air‐Conditioning Engineers.]

Playful vs. serious instruction giving in a 3D game environment

In this paper we introduce two NLG systems that we developed for the GIVE challenge, which was aimed at the evaluation of natural language generation (NLG) systems. The Challenge involved automatically generating instructions for users to carry out a task in a 3D game environment. One of our systems focused on generating optimally helpful ‘serious’ instructions while the other focused on entertainment, providing more playful instructions. We used the data gathered in the Challenge – both subjective user ratings and objective task performance data – to compare the efficiency and entertainment value of both systems. We found a clear difference in efficiency, but were unable to prove that one system was more entertaining than the other. This could be explained by the fact that the set-up and evaluation methods of the GIVE Challenge were not aimed at measuring entertainment. Based on our experiences, we give some suggestions for the set-up of future installments of the Challenge.

Behavioral Correlates of Working Memory Following Acute Aerobic and Resistance Exercise

There is growing interest regarding the effect of acute exercise on cognition. Recent evidence suggests that the positive acute exercise-induced changes in cognition are greatest for tasks or task components with larger executive control demands. The further investigation of the influence of other modalities of exercise may lead to a better understanding of the mechanisms responsible for exercise-induced changes in cognition. PURPOSE: We investigated whether acute bouts of aerobic versus resistance exercise would have a differential effect on the executive control of working memory. METHODS: Twenty-one young adult participants (9 females, age = 20.2 ± 0.3 years) completed a VO2max test and 1-Rep Max strength tests on 7 major muscle groups. On days 2, 3, and 4 participants completed a working memory task prior to the start of, immediately following, and 30 minutes following the experimental condition, which consisted of either 30 minutes of seated rest, resistance exercise (3 sets of 8 - 12 repetitions at 80% of their 1RM for each of the 7 major muscle groups), or aerobic exercise (60 - 70% of VO2max). Working memory was measured via a modified Sternberg task, which manipulates the amount of information required at encoding, while measures of reaction time and response accuracy were examined. RESULTS: Task performance data indicated shorter reaction time immediately (693.1 ± 21.2 ms) and 30 minutes (664.3 ± 14.9 ms) following acute aerobic exercise relative to the pre-exercise baseline (750.5 ± 19.0 ms), with a larger reduction in RT occurring for task conditions requiring increased working memory capacity. No effects were observed following resistance exercise or seated rest. CONCLUSION: These data extend the acute exercise-cognition database, indicating that changes in cognition following an acute bout of exercise are modality specific with a disproportionately larger influence for aspects of cognition requiring increased amounts of executive control.

The role of intervening variables in driver–ACC cooperation

This paper analyzes the behavior of drivers using Adaptive Cruise Control (ACC) within the theoretical framework of Human–Machine Cooperation. The study was carried out on a driving simulator. Driving task performance data and responses to a trust questionnaire were analyzed in order to examine the relationship between driver reliance on ACC and such intervening variables as trust, perceived workload and perceived risk. The participants were divided a posteriori into two groups according to their use of the ACC device during the experimental run. The results show that high-use drivers seemed to cooperate more with ACC than low-use drivers, who tended to perceive more risk and a higher workload. These findings are discussed in the light of Riley's theory of operator reliance on automation.

A computerized analysis of robotic versus laparoscopic task performance

Robotic technology has been postulated to improve performance in advanced surgical skills. We utilized a novel computerized assessment system to objectively describe the technical enhancement in task performance comparing robotic and laparoscopic instrumentation. Advanced laparoscopic surgeons (2-10 yrs experience) performed three unique task modules using laparoscopic and Telerobotic surgical instrumentation (Intuitive Surgical, Sunnyvale, CA). Performance was evaluated using a computerized assessment system (ProMIS, Dublin, Ireland) and results were recorded as time (s), path (mm) and precision. Each surgeon had an initial training session followed by two testing sessions for each module. A paired Student's t-test was used to analyze the data. Ten surgeons completed the study. 8/10 surgeons had significant technical enhancement utilizing robotic technology. The ProMIS computerized assessment system can be modified to objectively obtain task performance data with robotic instrumentation. All the tasks were performed faster and with more precision using the robotic technology than standard laparoscopy.

Brain Response Correlates of Decisional Capacity in Schizophrenia: A Preliminary fMRI Study

Brain Response Correlates of Decisional Capacity in Schizophrenia: A Preliminary fMRI Study

An Empirical Study of Relationships Between Traditional Usability

Task performance data and subjective assessment data are widely used as usability measures in the human-computer interaction (HCI) field. Recently, physiology has also been explored as a metric for evaluating usability. However, it is not clear how physiological measures relate to traditional usability evaluation indexes. In this paper, we investigate the relationships among three kinds of data: task performance, subjective assessment and physiological measures. We found evidence that physiological data correlate with task performance data in a video game: with a decrease of the task performance level, the normalized galvanic skin response (GSR) increases. In addition, physiological data are mirrored in subjective reports assessing stress level. The research provides an initial step toward using physiology as a complementary or an independent usability measure for HCI evaluation.

Neural Mechanisms of Cognitive Control: An Integrative Model of Stroop Task Performance and fMRI Data

We address the connection between conceptual knowledge and cognitive control using a neural network model. This model extends a widely held theory of cognitive control [Cohen, J. D., Dunbar, K., & McClelland, J. L. On the control of automatic processes: A parallel distributed processing model of the Stroop effect. Psychological Review, 97, 332-361, 1990] so that it can explain new empirical findings. Leveraging other computational modeling work, we hypothesize that representations used for task control are recruited from preexisting representations for categories, such as the concept of color relevant to the Stroop task we model here. This hypothesis allows the model to account for otherwise puzzling fMRI results, such as increased activity in brain regions processing to-be-ignored information. In addition, biologically motivated changes in the model's pattern of connectivity show how global competition can arise when inhibition is strictly local, as it seems to be in the cortex. We also discuss the potential for this theory to unify models of task control with other forms of attention.

Learning, life history, and productivity : Children's lives in the Okavango Delta, Botswana.

This article introduces a new model of the relationship between growth and learning and tests a set of hypotheses related to the development of adult competency using time allocation, anthropometric, and experimental task performance data collected between 1992 and 1997 in a multiethnic community in the Okavango Delta, Botswana. Building on seminal work in life history theory by Hawkes, Blurton Jones and associates, and Kaplan and associates, the punctuated development model presented here incorporates the effects of both growth and learning constraints on age-specific task performance. In addition, the payoff to investment in two forms of embodied capital, growth-based and learning-based, are examined in relation to features of the socioecology, including subsistence economy and family composition.The three main findings are:1.The development of adult competency in specific tasks entails a steplike relationship between growth- and experience-based forms of embodied capital in the ontogeny of ability acquisition.2.There is a trade-off between the acquisition of experience-based embodied capital in the form of skills and knowledge and immediate productivity among children. Time allocation to these alternatives is primarily determined by the short- and long-term costs and benefits to parents of investment in children's embodied capital.3.The availability of laborers and the overall labor requirements of the household are major determinants of investment in alternate forms of embodied capital and resulting variation in children's time allocation. The value of children's labor to their parents is dependent upon the opportunity costs to engaging in other activities not only for the child in question but also for potential substitute laborers. These results have important implications for our understanding of the role of growth and learning in the evolution of the human juvenile period, as well as for our understanding of cross-cultural variation in child growth and development and patterns of work and play.

Functional MRI in Epilepsy

*Department of Neurology, Medical College of Wisconsin, Milwaukee, Wisconsin, U.S.A.; †University Hospital Gent, Gent, Belgium; ‡Departments of Diagnostic Radiology and Neurosurgery, Yale University, New Haven, Connecticut; §Department of Neurology, University of Pennsylvania, Philadelphia, Pennsylvania; Clinical Epilepsy Section, National Institutes of Health, Bethesda, Maryland; ¶Department of Radiology, Mayo Clinic, Rochester, Minnesota; and **Department of Neurology, Medical College of Georgia, Augusta, Georgia, U.S.A.

Gender differences in the functional organization of the brain for working memory.

Gender differences in brain activation during working memory tasks were examined with fMRI. Seventeen right-handed subjects (nine males, eight females) were studied with four different verbal working memory tasks of varying difficulty using whole brain echo-planar fMRI. Consistent with prior studies, we observed activation of the lateral prefrontal cortices (LPFC), the parietal cortices (PC), and additionally, caudate activation in both sexes. The volume of activated brain tissue increased with increasing task difficulty. For all four tasks, the male subjects showed bilateral activation or right-sided dominance (LPFC, PC and caudate), whereas females showed activation predominantly in the left hemisphere. The task performance data demonstrated higher accuracy and slightly slower reaction times for the female subjects. Our results show a highly significant (p < 0.001) gender differences in the functional organization of the brain for working memory. These gender-specific differences in functional organization of the brain may be due to gender-differences in problem solving strategies or the neurodevelopment. Therefore, gender matching or stratification is required for studies of brain function using imaging techniques.

Executive Functioning in Preschool Children: Performance on A-Not-B and Other Delayed Response Format Tasks

The A-not-B (AB) task has been hypothesized to measure executive/frontal lobe function; however, the developmental and measurement characteristics of this task have not been investigated. Performance on AB and comparison tasks adapted from developmental and neuroscience literature was examined in 117 preschool children (ages 23–66 months). Age significantly predicted performance on AB, Delayed Alternation, Spatial Reversal, Color Reversal, and Self-Control tasks. A four-factor analytic model best fit task performance data. AB task indices loaded on two factors with measures from the Self-Control and Delayed Alternation tasks, respectively. AB indices did not load with those from the reversal tasks despite similarities in task administration and presumed cognitive demand (working memory). These results indicate that AB is sensitive to individual differences in age-related performance in preschool children and suggest that AB performance is related to both working memory and inhibition processes in this age range.

Development and evaluation of a microprocessor-based ergonomic dosimeter for evaluating carpentry tasks

This portable and self-contained lightweight microprocessor based Ergonomic Dosimeter is designed to collect continuously postural angles of the torso and the upper arm in the sagittal plane and the number of kneeling activities. Up to 4 h of task performance data can be stored in a non-volatile memory of the dosimeter, which can then be downloaded to a lap-top computer. The portable dosimeter was tested for test-retest reliability, compared with posture data obtained with a computer-based video analysis system and evaluated at a carpenter’s apprentices school and at a construction site. The dosimeter was shown to be suitable for collecting posture and kneeling data for a prolonged period at construction sites.

Military Job Analysis: A Historical Perspective

The U.S. military services pioneered the development of several job and occupational analysis methods and the application of these technologies. These job analysis techniques evolved to meet a variety of human resource planning and management needs, particularly reviewing classification structure and validating specialty initial skills training programs. A task inventory approach was useful for most military needs, and this approach was developed into a comprehensive occupational data analysis system. Task performance data (percentage performing each task, relative time spent) were later supplemented with task factor ratings (by a subset of experi- enced technical specialists), such as task difficulty and training emphasis recom- mended for the training of new personnel. As military budgets are increasingly constrained, these types of data, as well as new kinds of task and task module level information (such as associated skills, knowledges, equipment utilized, etc.), become critical for assisting military decision makers in modeling how to restructure occu- pations and optimally allocate scarce manpower and training resources. There is the potential for considerable general utility to the country as a whole in military and civilian use of such task and task module technologies for the study of jobs, occupations, and organizations.

Menu search and selection processes: a quantitative performance model

A criterion-based model is proposed that accounts for variations in search strategies and response accuracy in a computer menu search task. Factors considered by the model are (a) user-perceived relationships among target items sought and menu alternatives available for selection, (b) number of alternatives available for selection and (c) the probability of an omission situation where the target item is not subsumed under any of the alternatives available for selection. Results reported in Pierce, Parkinson and Sisson (1992) ( Int. J. Man-Machine Studies, 37, 000-000) showed that all three factors significantly influenced menu search and response accuracy. A data-fitting exercise is described in which search strategy and response accuracy data were fitted to model prediction functions by estimating best fitting values for model criteria. It is shown that processes suggested by the model are consistent with the majority of the findings obtained from analyses of menu task performance data.

Hand Controller Commonality Evaluation Process

Remotely controlled devices will be used extensively to support Space Station Freedom on-orbit assembly, maintenance, and payloads. These include crane-type or remote manipulator systems (RMS), dexterous robots, and remotely-piloted free flyers. A hand controller evaluation process has been established at NASA's Johnson Space Center to determine the appropriate hand controller configurations required to support these devices. Three test facilities include dynamic computer simulations, kinematic computer simulations, and physical simulations. The dynamic simulator supported a rate-controlled RMS and a free flyer. The kinematic simulator supported a rate-controlled RMS and a rate or position-controlled dexterous manipulator. The physical simulator supported a rate, position, or force-reflecting dexterous manipulator. Standard interfaces were developed to evaluate six different hand controllers in all three facilities. The hand controllers included six degree-of-freedom (DOF) position and rate mini-master and joystick controllers, and three-DOF rate controllers. There were six tasks and four non-astronaut subjects per task. All six controllers were tested for each task. Six astronauts then completed all tasks using all controllers for each task. Data collected included task performance data, subjective comments, and anthropometric data. The results of these evaluations were then used to make hand controller configuration recommendations to the Space Station Freedom Program.