Differences In User Performance Research Articles

Impact of the format of user instructions on the handling of a wrist blood pressure monitor.

Few studies have focused on procedural documents in the field of home medical devices, although incorrect use and usability problems can have important consequences for the patient's health. In this study, we focused on the procedural learning of a home medical device, a blood pressure monitor. Five formats (unimodal: text, audio, pictures; and multimodal: text/audio with pictures) were tested on 124 novice participants randomly assigned to 5 groups. We judged the quality of the formats on the basis of three metrics: efficiency (i.e., handling errors), effectiveness (i.e., consultation time of the procedure, execution time of the devices) and memorability (i.e., recall task). Results suggest that the audio format was more effective than the other groups but also the least efficient. We consider the audio format to be beneficial for patient safety because this format would oblige participants to use a strategy suited to the situation, namely an atomization of the action corresponding to a segmentation of information less likely to saturate working memory. Results in relation to the other formats did not show a more effective, efficient and memorable format. This lack of difference in user performance is nevertheless important, illustrating the need to adapt each instruction to the context of learning, i.e., according to the users, the environment, the resources and the complexity of the task to be executed.

Visual Search Performance and Strategy for Three-Dimensional Visualization Systems: Impact of Radiologist Experience

The visual search performance of radiologists is systematically challenged due to the use of flat-panel displays when viewing multi-slice images. Three-dimensional visualization of information is expected to facilitate visual search for radiologists. However, as the differences in user performance, depending on radiologist experience, have not yet been assessed for real three-dimensional images, it is difficult to estimate the usability of new visualization systems for radiologists with different amounts of professional search experience. In this study, we aimed to determine how three-dimensional visual search is affected by radiologist experience and the physical properties of stimuli. Radiologists, residents, and medical students accomplished visual search tasks with varying set sizes and target-distractor similarities. Search items were presented on multiple display planes and image manipulation was conducted using a computer keyboard. We compared the search outcome and strategy among three groups of participants. All three groups submitted a similar number of correct responses depending on target-distractor similarity. However, in contrast to radiologists and residents, it took a considerably longer time for students to find the targets. The major differences were evident in search strategy when comparing the image manipulations of radiologists with those of residents and students. Our results demonstrated that experienced radiologists’ search performance benefited from seeing the entire three-dimensional view of an image, whereas residents and students focused instead on reducing the amount of information and searched in a selective manner. These findings should be taken into account when planning the implementation of new workstations with three-dimensional visualization in radiology.

Star and polyline glyphs in a grid plot and on a map display: which perform better?

ABSTRACTGlyphs are small geometric shapes that in geovisualization are often used to represent multidimensional spatial data. The aim of this study is to investigate the effectiveness of their two types – star and polyline glyphs, as they can encode the same message and can provide similar functionality. Thus, if the two glyph types are similar and can be used for the same data, the question arises as to which of them better facilitates various user tasks. To address this question, an empirical study of 26 individual users is conducted to investigate differences in user performance for polyline and star glyphs shown either in a grid plot or on a map display. In this study, a task-based approach with eye-tracking is applied, as well as a subjective questionnaire and a psychological test of cognitive style. The finding is that polyline glyphs better facilitate tasks when datapoint values in glyphs are to be read, whereas star glyphs are better when a visual search among glyphs is to be done. Moreover, the results reveal that the map display works better than the grid plot. If star glyphs are to be used, the key (legend) needs to be better incorporated into a visual interface.

COMPARISON OF USER PERFORMANCE WITH INTERACTIVE AND STATIC 3D VISUALIZATION – PILOT STUDY

Interactive 3D visualizations of spatial data are currently available and popular through various applications such as Google Earth, ArcScene, etc. Several scientific studies have focused on user performance with 3D visualization, but static perspective views are used as stimuli in most of the studies. The main objective of this paper is to try to identify potential differences in user performance with static perspective views and interactive visualizations. This research is an exploratory study. An experiment was designed as a between-subject study and a customized testing tool based on open web technologies was used for the experiment. The testing set consists of an initial questionnaire, a training task and four experimental tasks. Selection of the highest point and determination of visibility from the top of a mountain were used as the experimental tasks. Speed and accuracy of each task performance of participants were recorded. The movement and actions in the virtual environment were also recorded within the interactive variant. The results show that participants deal with the tasks faster when using static visualization. The average error rate was also higher in the static variant. The findings from this pilot study will be used for further testing, especially for formulating of hypotheses and designing of subsequent experiments.

COMPARISON OF USER PERFORMANCE WITH INTERACTIVE AND STATIC 3D VISUALIZATION – PILOT STUDY

Abstract. Interactive 3D visualizations of spatial data are currently available and popular through various applications such as Google Earth, ArcScene, etc. Several scientific studies have focused on user performance with 3D visualization, but static perspective views are used as stimuli in most of the studies. The main objective of this paper is to try to identify potential differences in user performance with static perspective views and interactive visualizations. This research is an exploratory study. An experiment was designed as a between-subject study and a customized testing tool based on open web technologies was used for the experiment. The testing set consists of an initial questionnaire, a training task and four experimental tasks. Selection of the highest point and determination of visibility from the top of a mountain were used as the experimental tasks. Speed and accuracy of each task performance of participants were recorded. The movement and actions in the virtual environment were also recorded within the interactive variant. The results show that participants deal with the tasks faster when using static visualization. The average error rate was also higher in the static variant. The findings from this pilot study will be used for further testing, especially for formulating of hypotheses and designing of subsequent experiments.

Towards an automatic user profiling system for online information sites

Purpose – The purpose of this paper is to identify demographic differences based on how users interact with web applications. The research is needed to develop future systems able to adapt the representation of online information to the user’s specific needs and preferences improving its usability. The following question guides this quest: is there a direct relationship between age and/or gender and interaction? Design/methodology/approach – GOMS (goals, operators, methods, and selection rules) analysis was used to reduce complex interaction tasks into basic operators like pointing, dragging, typing, etc. An experiment was designed to analyse the user performance in the use of these operators through five complex tasks: point-and-click, drag-and-drop, text selection, text edition and menu selection. The sample comprises 592 individuals which took part in the experiment. The performance was analysed using multivariate regression analysis. User laterality and the the user experience were used as control variables. Findings – The factors studied are significant enough to support user classification. The analysis evidenced that men performed significantly better than women when executing interaction pointing and dragging GOMS’s operators, but no significant differences arose with regard to the performance in the typing operators. Older users performed worse in all the interaction tasks. No significant performance differences were detected between left and right-handed users. Research limitations/implications – The study pretends to lay the ground for developing artificial intelligence-based classification systems (e.g. neural networks, decision trees, etc.) able to detect significant differences in user performance, classifying users according to their age, gender and laterality. Practical implications – This user profiling would drive the organisation, selection and representation of the online information according to the specific preferences and needs of each user. This would allow the design of new personalisation algorithms able to perform dynamic adaptation of user interfaces in order to improve the usability of online information systems. Originality/value – This work extends previous research on user performance under a new approach and improved accuracy. First, it relies on the combined and simultaneous analysis of ageing and gender and the use of user laterality and experience as control variables. Second, the use of the GOMS analysis allowed the design of tests that closely resemble the user interaction in online information systems. Third, the size of the sample used in this analysis is much bigger than those used in previous works, allowing a more thorough data analysis which includes the estimation of an advanced model which is quantile regression.

Designing product feature upgrades: The role of user processing and design change

ObjectiveThis paper aims to examine user performance changes based on user processing level and changes made to a task during a product upgrade. BackgroundUser processing level is defined as the type of cognitive processing a user employs to complete a task and is identified as either controlled or automated. Changes to a product task are categorized into three types: omission (removal of a step), commission (addition of a step), and sequence (new order of steps). MethodParticipants completed tasks on a computer-based inventory management program. The tasks were designed to represent the three types of design changes and two levels of user processing. Example tasks included adding inventory to the system and auditing an inventory level. Performance time and errors were recorded. ResultsThe results indicate a difference in user performance based on processing type: participants adapted to changes in the program more readily when using controlled processing as opposed to automated processing. The variance of user performance was greater for tasks that required controlled processing. Design changes were most readily adapted to with a change of sequence. Tasks that were altered using omission and commission caused a greater negative effect on the participants’ performance. ConclusionWhen using automated processing, users can more readily adapt to design changes of sequence when compared to other types of design changes. Moreover, users’ recovery rates are higher when using controlled rather than automated processing. Relevance to industryWhen upgrading consumer products, designers should be mindful of the way tasks are changed, particularly for common automated tasks, which will allow users to more easily adapt to upgraded products.

User-centered virtual environment design for virtual rehabilitation

BackgroundAs physical and cognitive rehabilitation protocols utilizing virtual environments transition from single applications to comprehensive rehabilitation programs there is a need for a new design cycle methodology. Current human-computer interaction designs focus on usability without benchmarking technology within a user-in-the-loop design cycle. The field of virtual rehabilitation is unique in that determining the efficacy of this genre of computer-aided therapies requires prior knowledge of technology issues that may confound patient outcome measures. Benchmarking the technology (e.g., displays or data gloves) using healthy controls may provide a means of characterizing the "normal" performance range of the virtual rehabilitation system. This standard not only allows therapists to select appropriate technology for use with their patient populations, it also allows them to account for technology limitations when assessing treatment efficacy.MethodsAn overview of the proposed user-centered design cycle is given. Comparisons of two optical see-through head-worn displays provide an example of benchmarking techniques. Benchmarks were obtained using a novel vision test capable of measuring a user's stereoacuity while wearing different types of head-worn displays. Results from healthy participants who performed both virtual and real-world versions of the stereoacuity test are discussed with respect to virtual rehabilitation design.ResultsThe user-centered design cycle argues for benchmarking to precede virtual environment construction, especially for therapeutic applications. Results from real-world testing illustrate the general limitations in stereoacuity attained when viewing content using a head-worn display. Further, the stereoacuity vision benchmark test highlights differences in user performance when utilizing a similar style of head-worn display. These results support the need for including benchmarks as a means of better understanding user outcomes, especially for patient populations.ConclusionsThe stereoacuity testing confirms that without benchmarking in the design cycle poor user performance could be misconstrued as resulting from the participant's injury state. Thus, a user-centered design cycle that includes benchmarking for the different sensory modalities is recommended for accurate interpretation of the efficacy of the virtual environment based rehabilitation programs.

A User Study to Compare Four Uncertainty Visualization Methods for 1D and 2D Datasets

Many techniques have been proposed to show uncertainty in data visualizations. However, very little is known about their effectiveness in conveying meaningful information. In this paper, we present a user study that evaluates the perception of uncertainty amongst four of the most commonly used techniques for visualizing uncertainty in one-dimensional and two-dimensional data. The techniques evaluated are traditional errorbars, scaled size of glyphs, color-mapping on glyphs, and color-mapping of uncertainty on the data surface. The study uses generated data that was designed to represent the systematic and random uncertainty components. Twenty-seven users performed two types of search tasks and two types of counting tasks on 1D and 2D datasets. The search tasks involved finding data points that were least or most uncertain. The counting tasks involved counting data features or uncertainty features. A 4x4 full-factorial ANOVA indicated a significant interaction between the techniques used and the type of tasks assigned for both datasets indicating that differences in performance between the four techniques depended on the type of task performed. Several one-way ANOVAs were computed to explore the simple main effects. Bonferronni's correction was used to control for the family-wise error rate for alpha-inflation. Although we did not find a consistent order among the four techniques for all the tasks, there are several findings from the study that we think are useful for uncertainty visualization design. We found a significant difference in user performance between searching for locations of high and searching for locations of low uncertainty. Errorbars consistently underperformed throughout the experiment. Scaling the size of glyphs and color-mapping of the surface performed reasonably well. The efficiency of most of these techniques were highly dependent on the tasks performed. We believe that these findings can be used in future uncertainty visualization design. In addition, the framework developed in this user study presents a structured approach to evaluate uncertainty visualization techniques, as well as provides a basis for future research in uncertainty visualization.

The Role of User Processing and Task Change on Task Performance

The purpose of this research is to minimize the negative effects and maximize the benefits of the introduction of new technology to consumers. This paper examines the effect of user processing and product change on the acceptance of product upgrades. User processing is defined as the type of cognitive processing a user employs to complete a task, and is identified as either controlled or automated. Three types of product changes are examined: omission (removing a task step), commission (adding a task step), and sequence (changing the order of task steps). Both the short term and long term effects of these changes on user performance are examined. The platform utilized was an inventory management computer program. Task completion time was collected from 31 subjects. The results indicate a difference in user performance based on processing type: subjects adapted to changes in the program more readily when using controlled processing as opposed to automated processing. Users have difficulty recovering from a task change when they are using automated processing. When they are using controlled processing, the change is more readily accepted. The variability of their performance was greater when using controlled processing. Task changes were most readily adapted to when the change was that of sequence. Changes of omission and commission caused a greater negative effect on the subjects' performance.

Multimodal geographic information systems: Adding haptic and auditory display

AbstractThe goal of this study was to discover the feasibility of adding haptic and auditory displays to traditional visual geographic information systems (GIS). The experiment was conducted with 51 participants to explore the difference in user performance (task completion time and accuracy) and user satisfaction with a multimodal GIS system, which was implemented with a haptic display, auditory display, and combined display. The experiment consisted of a series of 36 tasks in which the participants were asked to identify the highest or the middle valued state among nine U.S. states on maps. The results showed that haptic displays produce faster and more accurate performance than auditory displays and combined displays for more complex tasks. In terms of user satisfaction, the participants preferred the combined display even though they performed best with the haptic display.

A primary care physician perspective survey on the limited use of handwriting and pen computing in the electronic medical record

Empirical research has demonstrated that simple design modifications in the electronic medical record (EMR) can improve user acceptance. Changes, such as pagination, the use of wireless pen slate computers and the use of digital ink (the graphic representation of the pen across the computer screen), can make dramatic differences in user performance, reduce uncertainty and increase acceptability. No survey has asked primary care physicians (PCPs) their preference of computer type for use during a consultation. Neither has any investigation determined whether or not physicians believed they could benefit from the use of handwriting (in the entry form of digital ink) as a supplement to the EMR. A survey was prepared to see if a group of PCPs in private practice was receptive to the use of digital ink, and what type of computer - desktop or wireless pen slate - would be preferred for use during a consultation. A wireless pen slate computer was described as having a screen large enough to display the same image seen on a desktop computer. Screen captures were used to demonstrate digital ink for handwriting and drawing. One-hundred-and-fifty-six of the 411 physicians responded (37.95%). Five physicians (3.2%) used a computer during a consultation. Ninety-nine (63.46%) would be willing to use a computer during a consultation. One-hundred-and-twenty-one (77.56%) indicated that digital ink would be useful as a supplement to the EMR. Of those who would use a computer during a consultation, 91/99 (91.09%) preferred a wireless pen slate computer to a desktop computer. Subgroup analysis indicated that those physicians who had been in practice more than 25 years or those who had low volume practices were less likely to desire a computer for use during a consultation. No subgroup showed less than 73% support for the use of handwriting in the EMR. Lack of standards (47.68%) and costs (40.40%) were chosen as the major reasons for not purchasing an EMR. Sixty-one PCPs (36.42%) indicated that a lack of typing skills was an issue and 39 (25.83%) said they had no time to learn. The use of handwriting in the EMR was broadly supported by this group of PCPs in private practice. Likewise, wireless pen computers were the overwhelming choice of computer for use during a consultation. In this group, older and lower volume physicians were less likely to desire a computer for use during a consultation. User acceptance of the EMR may be related to how closely it resembles the processes that are being automated. More surveys are required to determine the needs and expectations of physicians. The data also support other research studies that demonstrate the preference for handwriting and wireless computers, and the need for a limited, standardised and controlled vocabulary.

The impact of data models and task complexity on end-user performance: an experimental investigation

The purpose of this study was to investigate similarities and differences in the quality of data representations produced by end-users using the relational model (RM), the extended entity-relationship model (EERM), and the object-oriented model (OOM). By performing laboratory experiments using MIS major students, quality was evaluated on five constructs of a data model (i.e. entity/object, descriptor, identifier, relationship and generalization hierarchy) and six facets of a relationship (i.e. unary one-to-one, unary one-to-many, binary one-to-one, binary one-to-many, binary many-to-many and ternary many-to-many-to-many).The research focused on two major issues: data model design and data model conversion. The first issue investigated the differences in user performance between the RM, the EERM and the OOM. The second investigated the differences in user performance between the RM and the relational conversions of the EERM and the OOM models. For the first issue, EERM and OOM scored much higher than the RM in correctness scores of binary one-to-many and binary many-to-many relationships, but only the EERM led to significance. The RM and OOM scored much higher than EERM for unary one-to-one relationships, however, only the RM resulted in significance. The OOM required significantly less time for task completion than EERM. For the second issue, RM and the relational conversion of OOM scored significantly higher than the relational conversion of EERM for unary one-to-one relationships.

Is the Trackball a Better Input Device for the Older Computer User

This study investigated age-related differences in user performance and preferences using two widely available computer pointing devices, a mouse and trackball. Participants acquired on-screen targets of varying distance and size using point-and-click and click-and-drag motions. It was found that older adults took longer to complete movements than younger adults and did so to a greater extent for large amplitude movements, but that their movements were less variable. There was no age difference in movement time or variable error between the two devices. It has been hypothesized that the findings reflected the adoption of a device independent movement strategy by the elderly designed to compensate for age-related declines in motor control. Analysis of muscle activity (RMS) of the forearm flexor and extensor showed no age-related differences in muscle activity. Ratings of perceived extertion revealed that older adults reported higher levels of exertion than younger adults using the mouse during click-and-drag motions. In light of findings demonstrating that older adults have lower strength, this finding implies that the mouse requires a greater percentage of the elderlys' maximum force. These results taken together suggest that the trackball may be a better device for the elderly computer user especially when performing frequent, repetitive actions for prolonged periods of time.

A proposed index of usability: A method for comparing the relative usability of different software systems

Usability is becoming a more and more important software criterion, but the present usability measurement methods are either difficult to apply, or overly dependent upon evaluators' expertise. Based on human information processing theory, this study identified eight human factors considerations which are relevant to software usability. These considerations as well as the three stages of human information processing theory formed the framework from which our Purdue Usability Testing Questionnaire (PUTQ) is derived. An experiment was conducted to test the validity of PUTQ. The experiment result showed high correlation between PUTQ and the Questionnaire for User Interaction Satisfaction (QUIS version 5.5). In addition, PUTQ detected the differences in user performance between two experimental interface systems, but QUIS failed to do so.