Multiple Resource Theory Research Articles

Predictive mental workload modeling for semiautonomous system design: Implications for systems of systems

AbstractPredictive mental workload modeling is one established tool within the broad systems engineering activity of Human Systems Integration (HSI). Using system architecture as the foundation, this paper explores the use of Multiple Resource Theory to create representative workload models for evaluating operational system‐of‐systems (SoS) concepts. Through careful consideration of task demands, conflict generated between tasks, and workload mitigation strategies, informed design decision can improve overall human‐system performance. An example involving a single pilot controlling multiple remotely piloted aircraft (RPA) is presented to illustrate the use of workload modeling. Several observations are made that drive measurably excessive workload: multitasking, communications, continuously updating situational awareness and mission planning. In addition, three metrics are proposed for incorporating human workload analysis during system design. This technique has applicability across a wide range systems‐of‐systems and operational concepts involving complex human‐system interactions. © 2012 Wiley Periodicals, Inc. Syst Eng 15

A perpetrator's accent impairs witnesses' memory for physical appearance.

Sometimes witnesses to crimes must remember both a perpetrator's appearance and voice. Drawing upon multiple resource theory as well as previous findings that processing foreign-accented speech is more demanding than processing unaccented speech, we hypothesized that a perpetrator's accent can impair memory for his or her appearance. In Experiment 1, we used a secondary visual search task to demonstrate that processing an accented versus unaccented message demands more cognitive resources. In two additional experiments, we extended that result by showing that witnesses trying to encode information spoken by a perpetrator with an accent rather than no accent provided poorer physical descriptions of him and identified his voice less accurately. We also found that witnesses who heard a more versus less detailed message reported fewer correct details about the perpetrator's appearance (Experiment 2), and a more rather than less threatening message led to less accurate descriptions (Experiment 3).

A systematic review of the psychological literature on interruption and its patient safety implications

To understand the complex effects of interruption in healthcare. As interruptions have been well studied in other domains, the authors undertook a systematic review of experimental studies in psychology and human-computer interaction to identify the task types and variables influencing interruption effects. 63 studies were identified from 812 articles retrieved by systematic searches. On the basis of interruption profiles for generic tasks, it was found that clinical tasks can be distinguished into three broad types: procedural, problem-solving, and decision-making. Twelve experimental variables that influence interruption effects were identified. Of these, six are the most important, based on the number of studies and because of their centrality to interruption effects, including working memory load, interruption position, similarity, modality, handling strategies, and practice effect. The variables are explained by three main theoretical frameworks: the activation-based goal memory model, prospective memory, and multiple resource theory. This review provides a useful starting point for a more comprehensive examination of interruptions potentially leading to an improved understanding about the impact of this phenomenon on patient safety and task efficiency. The authors provide some recommendations to counter interruption effects. The effects of interruption are the outcome of a complex set of variables and should not be considered as uniformly predictable or bad. The task types, variables, and theories should help us better to identify which clinical tasks and contexts are most susceptible and assist in the design of information systems and processes that are resilient to interruption.

A perpetrator's accent impairs witnesses' memory for physical appearance.

Sometimes witnesses to crimes must remember both a perpetrator's appearance and voice. Drawing upon multiple resource theory as well as previous findings that processing foreign-accented speech is more demanding than processing unaccented speech, we hypothesized that a perpetrator's accent can impair memory for his or her appearance. In Experiment 1, we used a secondary visual search task to demonstrate that processing an accented versus unaccented message demands more cognitive resources. In two additional experiments, we extended that result by showing that witnesses trying to encode information spoken by a perpetrator with an accent rather than no accent provided poorer physical descriptions of him and identified his voice less accurately. We also found that witnesses who heard a more versus less detailed message reported fewer correct details about the perpetrator's appearance (Experiment 2), and a more rather than less threatening message led to less accurate descriptions (Experiment 3).

The effect of distractor modality and processing code on human-automation interaction

We examined the effect of distractor characteristics (modality and processing code) on visual search performance and interaction with an automated decision aid. Multiple Resource Theory suggests that concurrent tasks that are processed similarly (e.g. two visual tasks) will cause greater interference than tasks that are not (e.g., a visual and auditory task). The impact of tasks that share processing and perceptual demands and their interaction with human-automation interaction is not established. In order to examine this, participants completed two blocks of a luggage screening simulation with or without the assistance of an automated aid. For one block, participants performed a concurrent distractor task drawn from one of four combinations of modality and processing code: auditory-verbal; auditory-spatial; visual-verbal; visual-spatial. We measured sensitivity, criterion setting, perceived workload, system trust, perceived system reliability, compliance, reliance, and confidence. Participants demonstrated highest sensitivity when performing with an auditory-spatial secondary task. Automation compliance was higher when the auditory-spatial distraction was present versus absent; however, system trust was highest in the auditory-verbal condition. Confidence (when disagreeing with the aid) was also highest when the distractor was auditory. This study indicates that some forms of auditory ‘distractors’ may actually help performance; these results further contribute to understanding how distractions influence performance when operators interact with automation and have implications for improved work environment and system design.

Predicting operator mental workload using a time-based algorithm

Existing workload algorithms based on the Multiple Resources Theory (MRT) provide an effective approach for diagnosing operator overload caused by interference among concurrent tasks, however, their ability to handle overload in single task conditions is limited. We argue a time-based algorithm, developed on the Information Processing (IP) model of workload (Hendy, Liao, & Milgram, 1997), provides a viable solution to address this limitation. In this study, we proposed a new algorithmic implementation of the IP model in the context of task network modeling. The new algorithm was implemented in a JAVA program and tested on an existing model of a Bakan vigilance task. The results obtained from the new algorithm demonstrated the feasibility of this solution. By integrating resource-based and time-based algorithms, analysts will be able to diagnose more accurately system performance breakdowns caused by operator overload.

Bypassing the Bottleneck: The Advantage of Fingertip Shear Feedback for Navigational Cues

Multiple resource models suggest that there will be less interference between tasks that do not share the same processing resources or modalities. Most in-vehicle navigational systems rely on some type of auditory modality, which can be problematic in situations with high levels of background noise or when one is engaged in a secondary auditory task, such as talking on a cell phone. The haptic or tactile sensory modality may be an effective alternative for presenting important navigational information in a vehicle. The current study compared the effectiveness of auditory vs. tactile navigational instructions to the operator of a moving vehicle. Tactile shear cues were provided to the participants' index fingers as they gripped the steering wheel. These tactile cues provided directional information that coincide with the direction of the applied skin displacement. In single task driving, participants made accurate lane changes in response to both auditory and tactile instructions. In dual-task driving, where participants conversed on a cell phone, participants were less accurate with auditory instructions than for tactile instructions. This suggests that in some instances, presenting information via the tactile modality may bypass the bottleneck described by multiple resource theories when there is competition for the same resources.

Enhanced Auditory Menu Cues on a Mobile Phone Improve Time-Shared Performance of a Driving-Like Dual Task

The growing trend of using mobile phones and other in-vehicle technologies (IVT) while driving has spurred research on driver distraction, its effects and alleviation (Ashley, 2001; Young & Regan, 2007). The present study used a dual task in which 21 undergraduates navigated a mobile phone contact list for a target name (secondary task) while playing a computer game representative of driving (primary task). The phone menu was enhanced with two audio navigation cues: traditional text-to-speech (TTS) and spearcons (i.e., compressed speech). These cues were tested with and without visual display of the contact list. Spearcons in conjunction with TTS enhanced performance on the primary task while having no negative effect on the secondary task. Auditory menus reduced perceived workload and increased subjective ratings. Results are discussed in terms of multiple resources theory and practical mobile phone menu design.

Getting the Buzz: What's Next for Tactile Information Delivery?

As a result of continuous technological advances, information delivery is becoming multi-modal and complex. In many professions (e.g., air traffic control, combat operations) an abundance of information is delivered simultaneously over the visual and auditory sensory channels resulting in cognitive overload and leading to performance degradation over time. The Multiple-Resource theory suggests that offloading information from overtaxed sensory modalities to other modalities can reduce workload (Wickens, 2002). If properly implemented, tactile displays – i.e., devices used to present information to the user by stimulating the skin – may be a viable solution in reducing sensory and cognitive overload from the visual and audio channels. Shifting information to the tactile (touch) channel or judicious use of tactile information in conjunction with auditory and/or visual cues can lead to a reduction in cognitive and perceptual overload and an increase in positive performance outcomes. The primary objective of this panel is to discuss the most promising developments in tactile research and how the next steps can lead to new application areas or to specific products. The panelists – representing academia, the military, and industry, can collectively speak to diverse tactile information delivery methodologies, their respective applications, and challenges for the path forward. Some applications have already been realized in aviation, robotics, medicine, and commercial products and these will be described. Ultimately, this panel session is expected to inspire interest in tactile information delivery and to identify promising pathways for research leading to new application areas and specific application products.

An Eye Movement Analysis of the Effect of Interruption Modality on Primary Task Resumption

We examined the effect of interruption modality (visual or auditory) on primary task (visual) resumption to determine which modality was the least disruptive. Theories examining interruption modality have focused on specific periods of the interruption timeline. Preemption theory has focused on the switch from the primary task to the interrupting task. Multiple resource theory has focused on interrupting tasks that are to be performed concurrently with the primary task. Our focus was on examining how interruption modality influences task resumption.We leverage the memory-for-goals theory, which suggests that maintaining an associative link between environmental cues and the suspended primary task goal is important for resumption. Three interruption modality conditions were examined: auditory interruption with the primary task visible, auditory interruption with a blank screen occluding the primary task, and a visual interruption occluding the primary task. Reaction time and eye movement data were collected. The auditory condition with the primary task visible was the least disruptive. Eye movement data suggest that participants in this condition were actively maintaining an associative link between relevant environmental cues on the primary task interface and the suspended primary task goal during the interruption. These data suggest that maintaining cue association is the important factor for reducing the disruptiveness of interruptions, not interruption modality. Interruption-prone computing environments should be designed to allow for the user to have access to relevant primary task cues during an interruption to minimize disruptiveness.

The role of auxiliary assumptions in the falsification of ergonomics theories

Based on the accusations that some important ergonomics theories are not falsifiable, the present article reflects on the issue in a different way than has been previously presented by ergonomics researchers. This discussion highlights the importance of auxiliary assumptions in deriving testable predictions and emphasises the complications that result from understanding falsification. These arguments are made more concrete by considering them in the context of an important ergonomics theory that has been the target of falsification-related accusations – Multiple Resource Theory. Finally, the implications of this specific discussion for larger philosophical issues relevant to the falsification of ergonomics theories are explicated.

When Content Matters: The Role of Processing Code in Tactile Display Design.

The distribution of tasks and stimuli across multiple modalities has been proposed as a means to support multitasking in data-rich environments. Recently, the tactile channel and, more specifically, communication via the use of tactile/haptic icons have received considerable interest. Past research has examined primarily the impact of concurrent task modality on the effectiveness of tactile information presentation. However, it is not well known to what extent the interpretation of iconic tactile patterns is affected by another attribute of information: the information processing codes of concurrent tasks. In two driving simulation studies (n = 25 for each), participants decoded icons composed of either spatial or nonspatial patterns of vibrations (engaging spatial and nonspatial processing code resources, respectively) while concurrently interpreting spatial or nonspatial visual task stimuli. As predicted by Multiple Resource Theory, performance was significantly worse (approximately 5-10 percent worse) when the tactile icons and visual tasks engaged the same processing code, with the overall worst performance in the spatial-spatial task pairing. The findings from these studies contribute to an improved understanding of information processing and can serve as input to multidimensional quantitative models of timesharing performance. From an applied perspective, the results suggest that competition for processing code resources warrants consideration, alongside other factors such as the naturalness of signal-message mapping, when designing iconic tactile displays. Nonspatially encoded tactile icons may be preferable in environments which already rely heavily on spatial processing, such as car cockpits.

Workload in Human-Robot Interaction: A Review of Manipulations and Outcomes

The current study reviews the relationship between manipulations of teleoperator workload and task outcomes, using multiple resource theory as the underlying framework. Results indicated that controlling more than two platforms is detrimental to many performance indices (reaction time, error rate), but overall productivity improves. For studies that manipulated workload for a single robot task, visual demands were a limiting factor, and interventions which reduced visual demands improved performance. The review concludes with guiding principles for managing workload and improving teleoperator performance.

Workload in Human-Robot Interaction: A Review of Manipulations and Outcomes

The current study reviews the relationship between manipulations of teleoperator workload and task outcomes, using multiple resource theory as the underlying framework. Results indicated that controlling more than two platforms is detrimental to many performance indices (reaction time, error rate), but overall productivity improves. For studies that manipulated workload for a single robot task, visual demands were a limiting factor, and interventions which reduced visual demands improved performance. The review concludes with guiding principles for managing workload and improving teleoperator performance.

Compensatory Strategies for Managing the Workload Demands of a Multimodal Reserve Capacity Task

The purpose of the present study was to use a newly-developed measure of reserve attentional capacity to evaluate unitary versus multiple resource theories of attention. Participants performed a primary visual monitoring task and were presented with visual, auditory, and tactile secondary loading tasks. The data indicate that participants failed to maintain performance on the primary task, as instructed, in order to maximize overall performance. A significant difference was found on the basis of primary task condition, wherein performance on the visual monitoring task was significantly worse when paired with a visual task versus auditory or tactile tasks. Although the data do not preclude interpretation in terms of a unitary resource model, data trends offer potential support for multiple resource models.

The Effect of Interruption Modality on Primary Task Resumption

The majority of empirical papers investigating the effect of interruption modality on primary task resumption have been grounded in Multiple Resource theory; this theory stresses the benefits of cross-modal information presentation. Alternatively, Altmann and Trafton's (2002) Memory for Goals theory suggests that maintaining an association between the suspended primary task goal and relevant environmental cues is critical to the task resumption process. Using reaction time and eye movement measures, the theoretical predictions of these two frameworks were empirically examined to determine whether interruption modality influences primary task resumption.

Effects of Secondary Loading Task Modality on Attentional Reserve Capacity

The purpose of the present study was to use a newly-developed measure of reserve attentional capacity to evaluate unitary versus multiple resource theories of attention. Participants performed a primary visual monitoring task and were presented with visual, auditory, and tactile secondary loading tasks. The data indicate that participants maintained performance on the primary task, as instructed, and performed the secondary task with any remaining attentional reserve capacity. A significant difference was found on the basis of secondary task modality, wherein performance on the visual secondary task was significantly worse than that of secondary auditory and tactile tasks. This result was additionally supported by scores on a subjective workload questionnaire. Although the data do not preclude interpretation in terms of a unitary resource model, data trends offer potential support for a multiple resource model.

Multiple Resources and Mental Workload

The objective is to lay out the rationale for multiple resource theory and the particular 4-D multiple resource model, as well as to show how the model is useful both as a design tool and as a means of predicting multitask workload overload. I describe the discoveries and developments regarding multiple resource theory that have emerged over the past 50 years that contribute to performance and workload prediction. The article presents a history of the multiple resource concept, a computational version of the multiple resource model applied to multitask driving simulation data, and the relation of multiple resources to workload. Research revealed the importance of the four dimensions in accounting for task interference and the association of resources with brain structure. Multiple resource models yielded high correlations between model predictions and data. Lower correlations also identified the existence of additional resources. The model was shown to be partially relevant to the concept of mental workload, with greatest relevance to performance breakdowns related to dual-task overload. Future challenges are identified. The most important application of the multiple resource model is to recommend design changes when conditions of multitask resource overload exist.

Prioritizing Tasks in the Cockpit: A Review of Cognitive Processing Models, Methods of Dealing with Cognitive Limitations, and Training Strategies

Pilots must continuously assess, prioritize, execute, monitor, and terminate tasks to the best of their ability to safely and effectively complete the flight mission, often in time critical situations and in a dynamic environment. Limitations on pilot abilities to multitask are related to many factors described in the literature. To better understand and manage those limitations cognitive processes, such as single channel theory, multiple channel theory, and multiple resource theory are identified and related to cognitive limitations. The methods pilots use to deal with those cognitive limitations, including attention, workload, and task management are explored. Results from task prioritization specific training studies indicate multitasking may be trainable. Outcomes from studies reviewed can be used to inform design and implementation of training curricula. Some strategies for design and implementation of task prioritization training are given.

Tactons in multitask environments: The interaction of presentation modality and processing code

Tactons, or vibrotactile icons, have been proposed as a means to communicate complex concepts to users and to support multitasking in environments involving numerous visual and/or auditory tasks and stimuli. This study investigated the role of processing code in the interpretation of tactons while performing concurrent visual tasks in such environments. Participants decoded tactons composed of spatiotemporal patterns of vibrations – requiring spatial processing – and interpreted one of two types of visual task stimuli – requiring either spatial or categorical processing – in a driving simulation. Compared to single-task performance, there was a significantly larger dual-task performance decrement when the tacton task was paired with the visual task requiring spatial (as compared to categorical) processing. The findings are consistent with the assertion of Multiple Resource Theory that interference between concurrent tasks is greater when these tasks involve the same processing code. They illustrate how distributing task-related information across modalities is beneficial but not sufficient to avoid task interference. A direct implication of the findings is to avoid the use of spatiotemporal tactons in environments which rely heavily on spatial processing resources, such as car cockpits or flight decks.