Usability Lab Research Articles

The Design:Lab as platform in participatory design research

The notion of laboratory or simply ‘lab’ has become popular in recent years in areas outside science and technology development. Learning Labs, Innovation Labs, Usability Labs, Media and Communication Labs and even Art Labs designate institutions or fora dedicated to change and experimentation. Influenced by these currents, we use the expression ‘Design:Lab’ as a shorthand description of open collaborations between many stakeholders sharing a mutual interest in design research in a particular field. Many have reacted to the term ‘laboratory’ or ‘lab’ as foreign and awkward to design, and we as well as others have frequently used other metaphors like workshop, studio or atelier in design research. In this article we will argue that the laboratory metaphor is particularly suitable and useful for the Design:Lab, and we will give examples of how we have worked with the Design:Lab as a platform for collaborative inquiries and knowledge production based on design experiments.

Usability Evaluation of Web Mapping Sites

To identify the potential usability problems of Web mapping sites, four different sites were evaluated: Google Maps, MSN Maps & Directions, MapQuest, and Multimap. The experiment comprised a series of expert evaluations and user tests. During the expert evaluations, eight usability engineers and eight cartographers examined the Web mapping sites by paying attention to their features and functionality. Additionally, eight user tests were carried out by ordinary users in a usability laboratory. In all, 403 usability problems were identified during the trial and were grouped according to their severity. A qualitative description is given of these usability problems, many of which were related to search operations that the users performed at the Web mapping sites. There were also several problems relating to the user interface, map visualisation, and map tools. We suggest some design guidelines for Web mapping sites based on the problems we identified and close the paper with a discussion of the findings and some conclusions.

Effects of interaction design in digital libraries on user interactions

PurposeThis study aims to investigate the effects of different search and browse features in digital libraries (DLs) on task interactions, and what features would lead to poor user experience.Design/methodology/approachThree operational DLs: ACM, IEEE CS, and IEEE Xplore are used in this study. These three DLs present different features in their search and browsing designs. Two information‐seeking tasks are constructed: one search task and one browsing task. An experiment was conducted in a usability laboratory. Data from 35 participants are collected on a set of measures for user interactions.FindingsThe results demonstrate significant differences in many aspects of the user interactions between the three DLs. For both search and browse designs, the features that lead to poor user interactions are identified.Research limitations/implicationsUser interactions are affected by specific design features in DLs. Some of the design features may lead to poor user performance and should be improved. The study was limited mainly in the variety and the number of tasks used.Originality/valueThe study provided empirical evidence to the effects of interaction design features in DLs on user interactions and performance. The results contribute to our knowledge about DL designs in general and about the three operational DLs in particular.

The effect of need for cognition on search performance

AbstractWe examine in this paper how need for cognition (NFC), one of the personal characteristics, affects search performance in two aspects: effectiveness measured by precision and efficiency measured by search time. Forty participants were invited individually to an on‐campus usability laboratory for the study. Each was asked to perform two search tasks, make usefulness judgments on search results they obtained, and rate the task difficulty level upon finishing each task. Their NFC levels were collected by a standard psychologist‐developed NFC Scale. Factor analysis, T‐test and General Linear Model in SPSS were used to analyze the collected data. NFC was found to have effects on search effectiveness in general, but not on time (efficiency). Not significant though, there were tendencies that the interaction of task difficulty and NFC affected search effectiveness and efficiency. Results proved that NFC is a significant facet that could influence search performance. More studies are needed to further explore the interaction effect of NFC and the level of task difficulty/complexity. Further research will also examine the effect of elaborative processing of information in contributing to the better performance of people with high NFC level.

Evaluations of Physiological Monitoring Displays: A Systematic Review

The purpose of this paper is to present the findings from a systematic review of evaluation studies for physiologic monitoring displays, centered on empirical assessments across all available settings and samples. The findings from this review give readers the opportunity to examine past work across studies and set the stage for the design and conduct of future evaluations. A broad literature search of the literature from 1991 to June 2007 on PubMed and PsycINFO databases was completed to locate data-based articles for physiologic monitoring device display evaluations. The results of this search plus several unpublished works yielded 23 publications and 31 studies. Participants were faster detecting an adverse event, making a diagnosis or a clinical decision in 18 of 31 studies. They showed improved accuracy in a clinical decision or diagnosis in 13 of 19 studies and they perceived a decreased mental workload in 3 of 8 studies. Eighteen studies used a within subjects design (mean sample size 16.5), and 9 studies used a between group design (mean group size 7.6). Study settings were usability laboratories for 15 studies and patient simulation laboratories for 6 studies. Study participants were anesthesiologists or anesthesiology residents for 19 studies and nurses for 5 studies. The advent of integrated graphical displays ushered a new era into physiological monitoring display designs. All but one study reported significant differences between traditional, numerical displays and novel displays; yet we know little about which graphical displays are optimal and why particular designs work. Future authors should use a theoretical model or framework to guide the study design, focus on other clinical study participants besides anesthesiologists, employ additional research methods and use more realistic and complex tasks and settings to increase external validity.

The Live Usability Lab: Open access archives and digital repositories

The Live Usability Lab: Open access archives and digital repositories

Human Factors in Action: Getting "Pumped" at a Nursing Usability Laboratory

We describe our experience with a Nursing Usability Laboratory, where human factors design principles were applied to common nursing procedures. Our first step was to develop a human factors usability checklist. We then used this checklist while observing 11 nurses completing two standardized tasks on a simulated patient: (1) programming an insulin infusion and (2) programming a heparin infusion. We found that a usability checklist can help to uncover systematic error-provoking conditions in nursing tasks, that immediate improvements can be made in nursing training and practice and that participant nurses found the process useful. This paper will be of interest to any hospital seeking to enhance safety by applying human factors design principles.

Innovation et évaluation des technologies innovantes de la e-santé à Lille

The university hospital of Lille and the Lille school of medicine always encouraged the development of tehcnological researches. For this purpose, they supported technological research and innovation with the ITM, but also technological evaluation with « Evalab », the first « usability lab » in Europe dedicated to the assessment of healthcare computer applications. This research closely associates searchers, companies, but also the future users (mainly healthcare professionals but also patients) to improve the acceptability of the systems, to avoid errors and adverse effects and increase the patient's safety.

Who are the users of publicly reported cancer treatment wait times (WT)?

6043 Background: Greater participation by patients in health care decision-making and public concerns about WT led Cancer Care Ontario (CCO) to post radiation WT by cancer type and treatment centre on its website http://www.cancercare.on.ca anticipating patients and referring physicians (MDs) would use WT information to access facilities with shorter waits. The availability of more radiation treatment facilities in Southern Ontario provides patients and MDs with more options for location of treatment. Methods: The usefulness of this approach was evaluated by an on-line survey, patient focus groups, physician interviews and usability lab testing. Results: 1,043 on-line surveys were completed by patients/family members (35%), media (24%) and others, including health professionals (HP) and administrators (AD). HP found the information useful and clear but patients were less satisfied, wanting information on the effect of WT on their illness. 45 individuals (40 treated patients, 5 family) participated in 8 geographically dispersed focus groups. Facilitator-led conversations were recorded, transcribed verbatim and content grouped in themes by 4 researchers. Most patients were unaware of the CCO WT information. Patients indicated that MDs should have and use WT information and determine speed of access to care based on urgency of condition. Patients would accept MD advice to travel to a more distant treatment facility but patients questioned why WT existed and why health care system not managed more efficiently. 15 MD phone interviews indicated MDs were distrustful of WT data, did not use the web data for referral, preferring usual practice patterns. MDs were reluctant to share WT information with patients for fear of creating unnecessary anxiety. AD found data useful as a stimulus for performance improvement. Usability lab testing uncovered numerous user preferences for redesign of the site, including its content. Conclusions: Current Ontario patients and MDs are not ‘consumers’ of WT information but AD see value in this information for system improvement. As society increasingly uses the internet as an information resource, future patients and MDs will likely utilize WT information in the decision-making for location of care. Supported by Cancer Care Ontario and grant 03110 from the Change Foundation. No significant financial relationships to disclose.

Re‐engineering the usability‐testing process for live multimedia systems

PurposeThis paper seeks to present a comparative study of the traditional usability‐testing process and the re‐engineered usability‐testing process for live multimedia systems.Design/methodology/approachProvides an overview of current usability‐testing techniques and usability laboratory configurations, and identifies some gaps in the traditional usability‐testing approach.FindingsTraditional usability‐testing procedures are suitable for testing systems in the static environment but prove to be sub‐optimal in testing systems for dynamic (real‐time) environments.Originality/valueThe traditional set‐up is compared with innovative laboratory configuration, which consists of three computer systems: the test system in the middle augmented by two systems on either side that function as the scenario presenter and the data collection system. The re‐engineered usability‐testing process streamlined usability experiments and reduced the task completion times.

Integration of 3D motion analysis with the Mobile Usability Lab for biomechanical analyses of human-device interfaces

Integration of 3D motion analysis with the Mobile Usability Lab for biomechanical analyses of human-device interfaces

Stochastic sensors

Stochastic sensors have been the subject of considerable recent interest as a result of their ability for reconfigurable, rapid, multi-analyte detection as well as their potential for sequencing DNA. These sensors are created by placing a nanometre-sized pore in an insulating membrane and measuring the ionic transport through the pore in the presence of the molecules of interest. The magnitude, duration, and rates of occurrence of the resultant current blockades allow rapid determination of analyte concentration as well as discrimination between similar molecular species. The bulk of the work in this nascent field has been focused on biologically-based stochastic sensors using protein pores and lipid membranes, although hybrid and completely non-biological systems are also the subject of active investigation. I review this work as well as recent efforts to extend these systems and translate them into usable laboratory and clinical tools.

Usability Evaluation of a Gui Prototype for a Ventilator Machine

Information presentation on the monitor screen of ventilator machines affects nurses' response and decision-making during ventilation treatment. The purpose of this study is to evaluate whether a new GUI (graphical user interface) prototype, the so-called circular display prototype can make deviations easy to detect. A numerical display prototype was made and used as a reference display in the evaluation. Six task scenarios that involved parameter changes were selected to simulate a real situation under volume control (VC) mode during ventilation treatment. Usability tests with the two display designs were carried out in a usability laboratory. Twenty medical nursing students participated as test subjects in the usability tests. The objective results showed that the graphical circular display had an advantage over the numerical display in interpreting parameter changes, but not in reducing the error rates for detecting the number of parameter changes or for forming an overall picture of the patient's situation. Furthermore, the circular display prototype did not improve the detection time. Although the majority of the test subjects preferred the graphical circular display, the results implied that several aspects of this prototype should be improved in a future development study.

Synchronous computer mediated communication captured by usability lab technologies: new interpretations

Research on interaction has provided important insights on how second languages may be acquired through interaction, and on how second language learners use their second language in various settings and tasks. Research into computer mediated communication (CMC) in a second language learning context has been a logical outcome of interactionist research. In CMC interactionist research it is typically the case that the products of data collection and objects of analysis are linear transcripts or chat logs. While these transcripts may help us to better understand the nature of learners’ interactions, they are essentially one-dimensional in nature, not allowing one to coordinate learners’ actions (utterances, gestures, changes in body posture, etc.) with the language they create during interactions. In this paper, we report on the use of a usability lab (UL) in gathering data on synchronous CMC (SCMC) among non-native English dyads. This UL allowed us to capture video, audio, and screen capture for all SCMC sessions. We demonstrate that this methodological approach coupled with a proposed coding technique brings to light many nuances of SCMC interaction that are obscured when relying on printed chatscripts alone.

The design collaboratorium

The "design collaboratorium" is a new usability practice that has been developed in an action research project between three industrial usability labs and a university. The design collaboratorium has been developed as a reaction to the failing capabilities of classical usability methods to cope with ubiquitous technologies. It has borrowed elements from participatory design and developed them further to become useful in large-scale industrial development organizations. The design collaboratorium will be presented through examples from a joint project: a vision project concerning wastewater treatment technology. In light of the case, we will discuss the philosophy underlying the design collaboratorium in further detail: the collaboration between active participants, the role of design artifacts, and the room as a meeting ground. Finally we lay out the working method and propose directions for future usability competencies.

Overcoming the Lack of Screen Space on Mobile Computers

One difficulty for interface design on mobile computers is lack of screen space caused by their small size. This paper describes a small pilot study and two formal experiments that investigate the usability of sonically-enhanced buttons of different sizes. The underlying hypothesis being that presenting information about the buttons in sound would increase their usability and allow their size to be reduced. An experimental interface was created that ran on a 3Com Palm III mobile computer and used a simple calculator-style interface to enter data. The buttons of the calculator were changed in size between 4×4, 8×8 and 16×16 pixels and used a range of different types of sound from basic to complex. Results showed that sounds significantly improved usability for both standard and small button sizes – more data could be entered with sonically-enhanced buttons and subjective workload reduced. More sophisticated sounds that presented more information about the state of the buttons were shown to be more effective than the standard Palm III sounds. The results showed that if sound was added to buttons then they could be reduced in size from 16×16 to 8×8 pixels without much loss in quantitative performance. This reduction in size, however, caused a significant increase in subjective workload. Results also showed that when a mobile device was used in a more realistic situation (whilst walking outside) usability was significantly reduced (with increased workload and less data entered) than when used in a usability laboratory. These studies show that sound can be beneficial for usability and that care must be taken to do testing in realistic environments to get a good measure of mobile device usability.

How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews

Objectives: To describe techniques for retrieval and appraisal used by consumers when they search for health information on the internet.Design: Qualitative study using focus groups, naturalistic observation of consumers searching...

A usability focus for an HCI project

This paper discusses a way to introduce the concepts of usability and usability testing into an undergraduate human-computer interface class that does not require an elaborate or expensive usability lab setting. This approach gives students a more concrete understanding of the concepts that are covered in the course and engages them in a way that lectures cannot. An appendix includes the complete set of course topics as well as an outline of the project deliverables.

‘Televaluation’ of clinical information systems: an integrative approach to assessing Web-based systems

The World Wide Web provides an unprecedented opportunity for widespread access to health-care applications by both patients and providers. The development of new methods for assessing the effectiveness and usability of these systems is becoming a critical issue. This paper describes the distance evaluation (i.e. ‘televaluation’) of emerging Web-based information technologies. In health informatics evaluation, there is a need for application of new ideas and methods from the fields of cognitive science and usability engineering. A framework is presented for conducting evaluations of health-care information technologies that integrates a number of methods, ranging from deployment of on-line questionnaires (and Web-based forms) to remote video-based usability testing of user interactions with clinical information systems. Examples illustrating application of these techniques are presented for the assessment of a patient clinical information system (PatCIS), as well as an evaluation of use of Web-based clinical guidelines. Issues in designing, prototyping and iteratively refining evaluation components are discussed, along with description of a ‘virtual’ usability laboratory.

Integrating usability techniques into software development

Now merged into Siemens Medical Solutions Health Services, our company, formerly called Shared Medical Systems, creates clinical, financial, and administrative software for the healthcare industry. Like other medium to large companies, SMS had reached a scale and maturity level that required the development process to be documented, predictable, repeatable, measurable, and usable by the development groups. After much study and consideration, senior management committed to implementing a universal OO development methodology. Senior management recognized the need to improve customer satisfaction, which had always been high but needed to he better in an increasingly competitive market. Management saw the introduction of usability practices as a prime means to achieve this objective. So they began to place greater emphasis on usability-even to the point of building and staffing dual state-of-the-art usability labs. Our goal throughout the projects we describe was to combine the best OO analysis and design practices and usability techniques to create a powerful, unified way to develop software.