Visual Momentum Research Articles

Decision Support for Flexible Manufacturing Systems: Application of the Cognitive Systems Engineering and Ecological Interface Design Approach

We applied the cognitive systems engineering (CSE)/ecological interface design (EID) approach to the work domain of flexible manufacturing systems (FMS). Work domain analyses (WDA) and activity analyses were conducted. Previous concerns regarding the suitability of CSE for FMS are addressed. Principles of EID (i.e., direct perception, direct manipulation, and visual momentum) were applied in designing an ecological interface for a flexible manufacturing system (EcoFlex). The critical links between the products of a WDA and the content of an ecological interface (a key innovation of EID) are made explicit. The evaluations of this interface were very positive across a wide variety of traditional and novel dependent variables (please see the companion manuscript). The overall success of this research program clearly demonstrates that the CSE/EID approach is capable of being applied to FMS. Contributions of this manuscript include both theoretical insights (defining properties of the FMS work domain, concrete examples of the principles of EID, critical links between CSE and EID, resolution of concerns about the overall framework) and practical applications (specific design solutions for FMS and similar work domains, and successful examples of the CSE/EID approach which can be studied and leveraged by students).

Small screen-big information challenge for older adults: a study on visual momentum and gesture navigation

ABSTRACT The explosion of information and the miniaturisation of mobile devices seems to be an oxymoron. A large amount of information means a wider information space, and a smaller screen means a more limited display capacity. ‘Small screen-big information’ can often be disorienting, especially for older adults. Treating it as navigation in information space, we explored solutions in terms of spatial cognition and gesture motor control. First, we introduce a concept from cognitive psychology, visual momentum (VM). In terms of gesture control, we propose the device-based method that differs from traditional touch-based methods. To test their roles, two experiments were conducted among 30 older adults and comparatively tested 30 younger adults. Experiment 1 examined the contribution of high VM and a device-based method. Experiment 2 further examined an important property of view translation, smoothness, in touch-based and device-based methods. Results show that increasing the VM by adding grids and using a device-based method can help users find targets faster and more accurately, which is particularly effective for the older group. Meanwhile, differences in navigation strategies between touch-based and device-based methods were proposed. Finally, the smoothness of a device-based method in visual information processing is also demonstrated.

Visual Storytelling in Arab Virtual Public Spheres

Abstract Across the Arab world, increasing digitization, spiking young demographics, expanding media education programs and rising cultural liberalism seem to be inducing greater visualization in the virtual sphere. This perception of the image as a key facet of online communications has been especially noted in the Arabian Gulf, where grand cultural and media projects embracing media cities, virtual museums, knowledge parks, online video platforms and film festivals are seen as critically supporting the region by telling its story to the world. But in order for the Arab world to sustain this visual momentum in the virtual public sphere, it must address certain challenges arising from its postcolonial visual media policies, historically-entrenched oral communication legacies, state authoritarianism and narrow Islamic views of the image as taboo. In a global communications environment, Arabs have much to gain by harnessing the power of the image in virtual space to enhance their international engagement. The Gulf region has already made some breakthroughs in this regard by fostering more open cultural policies, supporting online visual content creation, embracing creative young talent, maintaining world-class telecommunications infrastructures and solidifying visual media education programs. But more has yet to be done to realize the full visual potential of the virtual sphere.

Cognitive untunneling multi-view system for teleoperators of heavy machines based on visual momentum and saliency

Abstract Work efficiency in unmanned construction is about 40% less than that in boarding operation owing to a lack of visual information. Thus, previous studies have developed methods to display rich visual information by adding multiple views. However, providing excess visual information can cause ‘cognitive tunneling’ in operators, which makes operators subconsciously attend to a limited view and ignore others. Therefore, we developed a cognitive untunneling multi-view system that enables operators to voluntarily select appropriate views depending on situations. The experimental results using a scale model indicate that the proposed view system could decrease work time, stoppage time and help operators switch views frequently and release their attention from a cab view compared with a conventional view system. These results suggest that the proposed view system can increase work efficiency and help operators avoid cognitive tunneling. In the future, we will develop a system to identify situations and change views automatically.

Ecological Interface Design for Computer Network Defense.

A prototype ecological interface for computer network defense (CND) was developed. Concerns about CND run high. Although there is a vast literature on CND, there is some indication that this research is not being translated into operational contexts. Part of the reason may be that CND has historically been treated as a strictly technical problem, rather than as a socio-technical problem. The cognitive systems engineering (CSE)/ecological interface design (EID) framework was used in the analysis and design of the prototype interface. A brief overview of CSE/EID is provided. EID principles of design (i.e., direct perception, direct manipulation and visual momentum) are described and illustrated through concrete examples from the ecological interface. Key features of the ecological interface include (a) a wide variety of alternative visual displays, (b) controls that allow easy, dynamic reconfiguration of these displays, (c) visual highlighting of functionally related information across displays, (d) control mechanisms to selectively filter massive data sets, and (e) the capability for easy expansion. Cyber attacks from a well-known data set are illustrated through screen shots. CND support needs to be developed with a triadic focus (i.e., humans interacting with technology to accomplish work) if it is to be effective. Iterative design and formal evaluation is also required. The discipline of human factors has a long tradition of success on both counts; it is time that HF became fully involved in CND. Direct application in supporting cyber analysts.

An Evaluation Method of Visualization Using Visual Momentum Based on Eye-Tracking Data

A new method based on eye-tracking data — visual momentum (VM) — was introduced to quantitatively evaluate a dynamic interactive visualization interface. We extracted the dimensionless factors from the raw eye-tracking data, including the fixation time factor [Formula: see text], the saccade amplitude factor D and the fixation number factor [Formula: see text]. A predictive regression model of VM was deduced by eye movement factors and the performance response time (RT). In Experiment 1, the experimental visualization materials were designed with six effectiveness levels according to design techniques proposed by Woods to improve VM (total replacement, fixed format data replacement, long shot, perceptual landmark and spatial representation) and were tested in six parallel subject groups. The coefficients of the regression model were calculated from the data of 42 valid subjects in Experiment 1. The mean VM of each group exhibited an increasing trend with an increase in design techniques. The data of the performance and eye tracking among the combined high VM group, middle VM group and low VM group indicated significant differences. The data analysis indicates that the results were consistent with the previous qualitative research of VM. We tested and verified this regression model in Experiment 2 with another dynamic interactive visualization. The results indicated that the VM calculated by the regression model was significantly correlated with the performance data. Therefore, the virtual parameter VM can be a quantitative indicator for evaluating dynamic visualization. It could be a useful evaluation method for the dynamic visualization in general working environments.

Improving the visual momentum of tethered viewpoint displays using spatial cue augmentation

A tethered viewpoint in an operator interface visualizes remote situations according to a robot’s heading direction. The tethered view, adhering to the line-of-sight requirement, results lower visual momentum when visualizing dense environments, such as during indoor teleoperation tasks. This problem occurs because the tethered view only partially visualizes spatial information, unlike the bird’s-eye view. Operators are thus inhibited to build their spatial mental model because the transitions happen as the robot moves. This paper presents an approach to improve the visual momentum of the tethered view by complementing the omitted spatial information. The approach augments simplified spatial cues on the excluded areas of a tethered view. The cues are purportedly used to illuminate the basic spatial structure of the surroundings and can thus help operators in constructing their spatial mental models. Evaluation of the presented approach was conducted using a simulated telerobot environment. The results indicated that the augmented view possessed a higher visual momentum, which exhibited by the lowered workload and enhanced levels of both spatial perception and situational awareness.

Veils

This paper briefly describes a prototype interface for computer network defense (CND) that has been developed using the cognitive systems engineering (CSE) / ecological interface design (EID) framework. This ecological interface is referred to as VEILS (Versatile Ecological Interface for Lockdown network Security). VEILS supports CND by collecting and integrating network defense information from a variety of sources including intruder detection / protection systems, firewalls, and system logs. VEILS was designed to be consistent with three principles of EID. The innovative displays represent the affordances of the work domain in a manner that is compatible with human visual processes (i.e., direct perception). The capability to specify both information content (different types) and information extent (scope) is supported entirely through intuitive and natural control input including point, click, and drag (i.e., direct manipulation). Finally, interface resources are used to choose alternative viewing perspectives, to highlight functional relationships across displays and to support effective navigation (i.e., visual momentum).

Time–Space Diagram as Controller Support Tool for Closed-Path Continuous-Descent Operations

Tactical control during a closed-path continuous-descent operation is unwanted because it reduces the aircraft’s ability to follow the optimized descent profile. To prevent tactical control, air traffic controllers apply arbitrarily large spacing buffers to ensure separation throughout the descent. A controller support tool is required for early deconfliction, spacing, and sequencing to facilitate these operations without the need to apply large buffers. The time–space diagram decision support tool was developed previously to make the constraints and complexity of a continuous-descent operation perceptually evident and provide tools and information to the controller to be an active problem solver. This paper addresses the further development and validation of the interface. The concept of visual momentum was applied to enhance the efficiency of working the multidisplay interface that consists of the plan view display and time–space diagram. Direct-manipulation tools were developed to enable the controller to plan and implement actions, such as speed and altitude control. A controller-in-the-loop-experiment was set up to validate the interface. In the experiment, subjects used either the time–space diagram support tool or, as a baseline, a stack list that provided the required spacing and the time to lose or gain. Both interfaces enabled the subjects to space the aircraft safely and efficiently. Compared to the baseline, the time–space diagram interface freed time to plan traffic ahead using the direct-manipulation interfaces, which (according to all subjects) worked intuitively. The number of instructions per aircraft was decreased by 35%. Early accurate speed control was applied, and use of heading vectors was no longer necessary in most scenarios. As a result, aircraft commenced their continuous descent at a higher altitude and greater distance from the runway. Controller workload was significantly reduced, and situation awareness was increased.

Evaluation of an Ecological Interface Design for Military Command and Control

Two alternative interfaces developed for military command and control were evaluated. The theoretical frameworks and concepts used during their development are discussed, and the findings are related to larger issues in display, interface, and system design. Key aspects of cognitive systems engineering (CSE) and ecological interface design (EID) are discussed. An ecological interface was designed with principles of direct perception, direct manipulation, and visual momentum. An experimental version of an existing interface was also developed. An experiment was conducted with a synthetic task environment that incorporated scenarios of tactical operations. Participants were experienced army officers. Dependent variables included status reports for friendly and enemy resources and activities, subjective workload, and information access. Significant results favoring the ecological interface were obtained for six of seven dependent measures. The ecological interface was easy to learn, easy to use, and dramatically more effective than the existing interface. The results are interpreted from the CSE-EID perspective, but insights from naturalistic decision making and situation awareness are also described. The specific design features of the ecological interface are directly applicable to military command and control and similar domains; the overall CSE-EID approach is applicable to interface design for all work domains.

Visual momentum redux

Over 25 years ago Woods (1984) introduced the concept of visual momentum: the extent to which an interface supports a practitioner in transitioning between various information-seeking activities that are required for understanding and exploring work domains. Increasing visual momentum requires the consideration of a range of “cognitive couplings” that span all levels of the interface: between multiple screens, within individual screens, and within a display on a screen. Although the concept has been well received, we believe that its potential to improve the quality of human computer interaction may be under-appreciated. Our purpose in this review is to provide a better understanding of visual momentum: to provide concrete and diverse examples of its successful application, to review empirical findings, to refine and expand the original design techniques that were proposed, and to integrate diverse terms that appear across different research communities.

Viewpoint Tethering for Remotely Operated Vehicles: Effects on Complex Terrain Navigation and Spatial Awareness

The effect of viewpoint on the navigation of complex terrain and on spatial awareness was examined with the use of a simulated remotely operated vehicle. The ability to build terrain models in real time may soon allow remote vehicular control from any viewpoint. A virtual tether couples the viewpoint to the vehicle's position and orientation, but shows more of the terrain than a fully immersive egocentric display. In this sense, it provides visual momentum by providing a view that incorporates egocentric and exocentric qualities. For this study, 12 participants navigated a simulated vehicle across complex virtual terrain using five different display viewpoints: egocentric, dynamic tether, rigid tether, 3-D exocentric, and 2-D exocentric. While navigating, participants had to avoid being seen by simulated enemy units. After the navigation task, participants' spatial awareness was assessed using a recognition task. The tethered displays minimized the time during which the participant's vehicle was visible to enemy positions. The egocentric display was more effective than exocentric displays (2-D or 3-D) for navigation, and the exocentric displays were more effective than egocentric for time seen during navigation and the recognition task. The tethered displays produced intermediate results for navigation and recognition. Viewpoint tethering produced the most effective displays for minimizing time seen, but tethered displays were less effective than egocentric and exocentric displays for navigation and recognition, respectively. A tethered display is recommended for applications in which it is necessary to understand the relation of nearby locations to one's own location.

Smooth Rotation of 2-D and 3-D Representations of Terrain: An Investigation Into the Utility of Visual Momentum

The potential advantage of visual momentum in the form of smooth rotation between two-dimensional (2-D) and three-dimensional (3-D) displays of geographic terrain was examined. The relative effectiveness of 2-D and 3-D displays is task dependent, leading to the need for multiple frames of reference as users switch tasks. The use of smooth rotation to provide visual momentum has received little scrutiny in the task-switching context. A cognitive model of the processes involved in switching viewpoints on a set of spatial elements is proposed. In three experiments, participants judged the properties of two points placed on terrain depicted as 2-D or 3-D displays. Participants indicated whether Point A was higher than Point B, or whether Point B could be seen from Point A. Participants performed the two tasks in pairs of trials, switching tasks and displays within the pair. In the continuous transition condition the display dynamically rotated in depth from one display format to the other. In the discrete condition there was an instantaneous viewpoint shift that varied across experiments (Experiment 1: immediate; Experiment 2: delay; Experiment 3: preview). Performance after continuous transition was superior to that after discrete transition. The visual momentum provided by smooth rotation helped users switch tasks. The use of dynamic transition is recommended when observers examine multiple views of terrain over time. The model may serve as a useful heuristic for designers. The results are pertinent to command and control, geological engineering, urban planning, and imagery analysis domains.

Effects of Transitioning between Perspective-Rendered Views

Increasingly, multiple views on the same scene may be made available via a network of sensors in security and surveillance applications. Similarly, virtual reality and gaming applications may provide a user with more than one view on a scene in an attempt to improve user orientation. Previous work has demonstrated the utility of ‘visual momentum’ by showing an advantage in spatial judgments when users are provided with a transition between 2D and perspective views. However, no prior research has examined the utility of smooth transitions between different perspective-rendered views of the same scene. The present experiment compared two types of smooth transition between perspective-rendered views, and demonstrated that transitions generally provide an advantage on a spatial judgment task. Notably, transitions between perspective views did not need to include a bird's eye view of the scene in order for users to make improved line-of-sight judgments.

The Effect of Visual Momentum on Learning Hierarchical Menu Structures in Small Displays

Navigating through menu structures to access functions and search for information on small displays in cellular phones and other handheld devices is difficult. Visual momentum is a display technique that may help users integrate information across successive displays, and thus improve navigation performance for devices with small screens. This study investigated the effects of visual momentum and menu structure on the learnability of menus on small displays. Learnability was assessed using a trials-to-mastery measure. Mastery criteria regarding search time and errors were set during a pilot study and the number of trials it took for participants to meet these criteria during the Main Experiment were recorded. It was found that providing preview information in the display did not help users learn to navigate through hierarchical menus, while providing motion in the display did help.

Advantage for Visual Momentum Not Based on Preview

Previous research has indicated that smooth rotation of geographic terrain between two- and three-dimensional (2D and 3D) views aids task switching. However, the time taken to show the smooth rotation may also provide a terrain preview for a post-rotation judgment. To test this possibility, we examined a situation where preview was provided but smooth transition violated. Twenty-four participants made judgments about the properties of two points placed on 2D or 3D displays of terrain. Participants performed the tasks in pairs of trials, switching tasks and displays between trials. In the continuous transition condition, the display rotated in depth and in azimuth from one display format to the other. In the discrete transition condition, the azimuth rotation was in the opposite direction, and then the terrain ”snapped” to the final orientation. The results showed that response time after transition was less for the continuous condition. We argue that smooth transition to the correct position provided improved visual momentum between displays.

Visual Momentum and Task Switching with 2D and 3D Displays of Geographic Terrain

We were interested in determining if the visual momentum provided by gradual transition between two- and three-dimensional (2D and 3D) views of geographic terrain aided task switching. Twenty-two participants made judgments about the properties of two points placed on 2D or 3D displays of terrain. Participants performed the tasks in pairs of trials, switching tasks and displays between trials. On half the trials (continuous transition), the display dynamically rotated in depth from one display format to the other. On the other half (discrete transition), participants were immediately shown the alternate display format. The results showed that response time after transition was less for the continuous condition, and that accuracy was greater for the continuous condition, especially for the 3D display. We argue that this was because the continuous transition provided improved visual momentum between consecutive displays, and recommend the use of dynamic transition when switching views on geographic terrain.

Tactical Displays for Combat Awareness: An Examination of Dimensionality and Frame of Reference Concepts and the Application of Cognitive Engineering

Three prototype displays are contrasted for aircraft navigation and tactical hazard awareness: a conventional 2-D coplanar display, an exocentric (God's eye) 3-D display, and an immersed (pilot's eye) 3-D display coupled with a small scale global display. Strengths and weaknesses of each display type are discussed, related particularly to the ambiguity of perspective displays and to attention allocation problems with dual-display suites. In Experiment 1, 8 pilots flew with each of the displays around an airspace populated by waypoints, hazards, and traffic. Performance measures revealed the anticipated costs and benefits of each display type. In Experiment 2, cognitive engineering techniques of performance and cognitive task analysis were applied to remediate the weaknesses of each display type. Emphasis was placed on display enhancements to resolve 3-D ambiguity and visual momentum and cueing techniques to facilitate attention allocation and switching. Analysis of performance of 27 additional participants in flight path tracking and hazard awareness measures revealed success in application of the cognitive engineering principles, greatly attenuating the weaknesses of each display.

Factors Influencing Spatial Awareness and Orientation in Desktop Virtual Environments

This study investigated influences on human spatial orientation and representation resulting from manipulations in the geometric field of view (GFOV) of a perspective display. Also examined was the impact of incorporating symbolic virtual environment enhancements in the interface based on visual momentum (VM) techniques. Sixty participants, pretested for spatial ability, were required to navigate through a virtual office building while performing a variety of spatial orientation tasks. A 3 × 2 × 2 mixed-subjects design compared three levels of GFOV, two levels of VM, and two levels of Difficulty. The findings indicate that decreases in GFOV produce spatial orientation and representation errors, and as task difficulty increases, errors are more pronounced. Furthermore, VM was shown to alleviate errors in spatial orientation and representation, especially when task difficulty was increased. Design recommendations are discussed based on the implications of the results.

Electronic Map Evaluation in Simulated Visual Meteorological Conditions

This study examined the effects of planar versus perspective electronic map displays during a simulated visual approach to landing. Thirty pilots were divided into 3 groups of 10 and assigned to 1 of 3 display conditions: 2-dimensional, 3-dimensional, or a 2-dimensional condition augmented with a "wedge" to provide a visual momentum between the display view and that in the forward field of view. Each display condition was manipulated in terms of frame of reference, either rotating track-up or fixed north-up, and assessments were made of the pilots' ability to maintain the flight path and awareness of surrounding terrain. For all display conditions, the track-up format supported the best tracking performance and also reported faster responses to some of the situation awareness tasks. The addition of a wedge onto the 2-dimensional display revealed a benefit in the form of improved lateral tracking with the north-up format. The results are discussed in terms of the costs and benefits associated with varying frame of reference and dimensionality on tasks of local guidance and global situation awareness.