Human Factors Testing Research Articles

Individual Differences in Computerized Test Performance for Systems Integration in Cockpit Management

Assessments of cognitive performance profiles, personality variables, cognitive workload relationships, and physiological indices suggest that there may be important individual differences in cognitive test performance and style during the execution of cockpit management tasks. Stability of cognitive performance profiles during repeated measures in a flight simulator and in human factors test facility suggest significant subject effects on performance of nine computerized cognitive tests within a complex cognitive assessment battery. Highly significant test effects were demonstrated and characterized by individual differences in performing them during the execution of cockpit management tasks. Results showed important clustering of the cognitive tests (Following Directions, Mark Numbers, Numbers and Words, Word Anagrams, Missing Items, Route Planning, Tower Puzzle, Information Purchase, and Logical Relations) in which varying clusters differed significantly from other clusters and patterns. Cognitive performance on some tests were related to assessments of personality, physiological indices, cognitive workload, and proficiency in monitoring instrument displays. For the integration of advanced cockpit technology and human cognitive performance capabilities in advanced cockpit management systems, inferences may be drawn which describe the types of cognitive performances which may be efficiently integrated with automated instrumentation in the cockpit. The design and integration of performance management tasks for the advanced cockpit includes consideration of the individual differences in performance of specific cognitive tests and the proficient performance of their components in the management system.

Multimessage Fiber‐Optic Lane‐Control Signs for Freeway Applications

Four prototype multimessage fiber‐optic lane control signs were evaluated for the Ministry of Transportation of Ontario to determine their suitability for freeway‐lane‐control‐signing applications. The evaluation consisted of three separate investigations, as follows. In the field test, the signs were situated and operated in an outdoor environment for approximately one year. Control instructions for cycling through each of 11 messages, varying brightness levels, and implementing a flashing mode were generated by an IBM‐PC‐compatible microcomputer using software specifically developed for this application. In the human factors test, a detailed questionnaire completed by volunteers was used to perform a statistical assessment of the visibility, and comprehensibility of each sign display. In the laboratory test, a laboratory evaluation was performed to measure various electrical and optical properties of the signs. The evaluation concluded that fiber‐optic technology can successfully be applied to lane‐spec...

Panel Discussion of “HRA — Where Shouldst Thou Turn?”: A Controversial View

The area of Human Reliability Analysis (HRA) has become increasingly important as both the nuclear industry and other high-technology industries have become conscious of the need for risk management and risk analysis. Techniques for performing and validating HRA should clearly be subjects of Human Factors Test and Evaluation attention. However, these issues have received less attention than they probably deserve from the Human Factors Society, despite the key contributions to the field of such HFS Test and Evaluation Group members as Dr. Alan Swain and Dr. David Meister. Issues that should be of interest to the HFS membership were recently raised in a white paper, recently published in “Reliability Engineering and System Safety” as an editorial. This editorial, titled “HRA — Where Shouldst Thou Turn?”, was written by Mr. Ed Dougherty, an HRA practitioner. The editorial takes to task (“pillories”, might not be too strong a term) almost every major technique and method for HRA currently in use, including his own. In addition, he castigates the US Nuclear Regulatory Commission attitudes and philosophies toward HRA use. Due to the controversial nature of the editorial, the journal editor, Dr. Apostolaikis, provided an opportunity for response to some fifteen practitioners/theorists of those mentioned. Some of these responders included such HFS members as Alan Swain, David Woods, Barry Kantowitz, Thomas Ryan, and Jens Rasmussen. However, these responders had no chance to review each others responses, nor has Mr. Dougherty had any opportunity to respond to the responses.

A Touch-Only User Interface for a Medical Monitor

A touchscreen interface was tested as the exclusive means for interacting with a computer-based monitor for the hospital intensive care environment. The use of touch in the medical environment combined with the fact that it was the sole means of user communication presented several human engineering challenges. Human factors testing of the design used 75 hours of mockup review by 38 clinical and administrative staff. In addition, 680 hours of field trial testing in the intensive care environment by 39 hospital staff were performed. The minimum size of visual and touch-sensitive target areas and the proximity of adjacent areas was determined. The touch recognition algorithm was modified to more accurately recognize near-edge targets. The use of a touch-sensitive QWERTY keyboard for patient name and id entry was shown to be acceptable. N-key rollover was deemed not viable with a touch interface in the critical care area. A swivel mount was introduced to compensate for different height and handed end-users.

Human Factors Test and Evaluation of the Peacekeeper (MX) ICBM

Test and evaluation is an essential part of any Human Factors involvement in system development. The role and methods of Human Factors in the Peacekeeper Weapon System Test Program are discussed. Factors contributing to the success of this effort are presented. These factors are applicable to any such program.

Display, Control, and Software Evaluation Requirements for Emerging Tactical Army Computer Systems

The soldier-computer interface (SCI) has become increasingly the principal focus of the man-machine interaction. As such, it represents a critical element to be assessed during human factors engineering test and evaluation. Inadequate design of displays and input/output devices such as keyboards can degrade system performance below levels necessary for functional effectiveness. Delays and errors that occur at the operator's console are translated directly into reduced system effectiveness. Adequate human factors test and evaluation requires the development and application of a sound methodology and criteria for testing and evaluating the effectiveness of display and input-output devices of the soldier-computer interface. This paper reviews the development of the methodology and criteria used to assess the SCI by the U.S. Army Test and Evaluation Command (TECOM).

Connecting theory and practice

This paper describes a case study of the Human Factors design, development, and testing of a computer-based financial analysis package. The project applied the “usability goals” method proposed by Bennett (1984) to structure the definition, design, and testing of the new system. Learnability was defined as a key attribute in the product concept because of its salience in users' perception of system quality. The learnability attribute was assigned an operational definition in terms of time to mastery and error avoidance/recovery. The “back-to-front” strategy of Didner & Butler (1982) was applied for designing the menus. Empirical testing of user performance on sample problems in the alpha stage indicated that the new system surpassed the learnability objective. Lessons learned from this case study concern leverage in getting better managerial attention for Human Factors considerations in development projects, and clearer structure to direct needed research.

Human Factors Testing of Icons for PC JR'S Word Processor Application

Problem: This paper describes a study which was done during the development of the PCjr's HomeWord word processor. The objective was to determine the extent to which menu icons were intuitively obvious. Method: Study Phase 1: 24 IBM employees were tested. Half of the subjects with WP experience and half of those without WP experience were given 10–15 minutes of “training” (operational definitions). The test consisted of two blocks of trials separated by a 15 minute rest period during which a demographic questionnaire was given. Subjects observed the icons on the screen and then completed a matched-pair multiple choice test. Included in the choices was a false choice to minimize guessing. Subjects' confidence level and subjective responses were also obtained. A confusion matrix was developed to identify which choices were made for each icon. Those icons which fell below a 90% correct identification cut-off were analyzed further. The Fisher Exact Probability Test was used to determine whether the correct selection of an icon and its function was dependent upon WP experience or the abbreviated “training”. Comments of the subjects were categorized and tabulated. Study Phase 2: 31 dependents of IBM employees, aged 8–18, were tested with icons representing a subset of basic WP tasks. The test design was the same as that for adult testing except that (1) all subjects were trained and tested in the same way, (2) training was more extensive, (3) no false choice was in the test instrument, and (4) confidence level data was eliminated after the pilot test. The test data were analyzed as for the adult testing except that (1) no confidence level data was used, and (2) a regression line was fit to identify icons vs. age. Comparisons were also made between test data for adults vs. minors. Results: Major results indicated that displaying all icons in a menu enhanced learning; the original design of several critical icons, e.g., Save Document vs. Delete Document, were confusing; and effective learning of the icons began at about 10 years of age. Conclusion: The improved icon designs added significantly to the salability of the product by eliminating confusing menu icons. Acknowledgement: This is to acknowledge the support of Mr. J. Lewis in the analysis and reporting for Study Phase 1.

Timely Application of Advanced Human Factors Test and Evaluation Techniques during the Acquisition of New Air Force Systems

To ensure that all major Air Force systems are subject to adequate human factors examination, the dependence on data-free critical analysis concerning system (and human) performance should be kept to a minimum during testing. Human factors test and evaluators (T&Eers) require the timely application of state-of-the-art (SOA) test and evaluation (T&E) techologies to keep pace with advanced system design. The Air Force needs to establish high-level policies directing human factors technology to ensure there are: 1) SOA T&E procedures, models and techniques; 2) appropriate human performance criteria; 3) current specifications, standards and handbooks; 4) qualified and trained T&Eers 5) SOA T&E instrumentation and equipment, and 6) SOA T&E software. A proposed model for transfer of human factors technologies for test and evaluation is presented.

Tools for the Development of Systems for Human Factors Experiments: An Example for the SSA

For a period of two years, Bolt Beranek ar Newman Inc. (BBN) worked with the Social Security Administration (SSA) to develop tools to enable exploration of human factors issues in the on-line collection and manipulation of data which are collected during interviews with clients applying for social security benefits. As part of this project, BBN developed a Human Factors Test and Evaluation Facility (TEF), wrote several systems with which human factors experiments were performed in that facility, and also produced a set of on-line tools. These tools allowed SSA personnel to specify parts of the system such as definitions of the data to be collected (complex data types and structures), simple constraints which the data must satisfy, and presentation mechanisms (e.g., two-dimensional forms). The experiments that were conducted, the features of the on-line tools which allowed relatively naive users to specify the complex behavior of the systems used for the experiments, and certain features of those systems that were developed with human factors considerations in mind are discussed.

Poison prevention packaging act, 1970: A human factors standard

This paper, the first of two, gives an account of legislation and regulations made in the USA requiring hazardous pharmaceutical and other household products to be packaged in child-resistant containers. Human factors test procedures and standards, in terms of which child-resistance is defined, are described. An account is given of those hazardous substances which regulations in the USA require should be packed in child-resistant containers. The paper concludes with a description of the effects of the regulations and of child-resistant containers in reducing mortality and morbidity associated with the ingestion of poisonous substances, particularly aspirin, in the USA. The second paper will briefly describe the effects of legislation made in England and Wales requiring certain pharmaceutical products to be packaged in child-resistant containers. It will also give an account of experiments, carried out in England and Sweden, describing the difficulties and inconvenience which the elderly and disabled experience when they attempt to use such containers.

Sensory Enhancement Technology for the Visually Handicapped

Computer technology is commonplace in todays work environment. Information management capabilities have recently been extended to visually handicapped personnel through the implementation of a Sensory Assistance Center at the Department of Justice. This paper discusses the analyzes and integration of human, mechanical and environmental components comprising the Center and the rigorous human factors testing of enhancement devices for special populations. Special user needs were identified and these findings applied to operating capabilities and limitations of the components in the Sensory Assistance Center.

Quantifying Operator Preference during Human Factors Test and Evaluation

The present paper will describe the construction and pilot testing of a human factors evaluation instrument. The instrument was constructed using psychometric procedures generally applied to development of attitude scales. The goal of the instrument was the quantification of operator preference in helicopter design within four major areas of human factors consideration: handling qualities, comfort/discomfort, human engineering design and safety. Each area had a common scale against which 10 specific parameters were evaluated. The ten items within each area were chosen using system operators' expertise, human factors standards, human factors experimental literature and the experience of the authors. The instrument was validated using a sample of 16 aviators during the conduct of an actual operational test. The results of these validation studies indicated a test-retest reliability of .85 (P < .001) and an inter-rater reliability of .93 (P < .001). Use of the instrument did provide statistically significant differences among aircraft candidates under various operational test conditions as measured within each of the four indices within the instrument. It is concluded that the instrument provides an effective method of quantifying the preference of operational aviators. The scrupulous development process using inputs from experienced aviators, experts in aviation field test design, and experts in test construction has provided a high measure of construct validity to the instrument.