WIMP Interface Research Articles

Mobile and Multimodal? A Comparative Evaluation of Interactive Workplaces for Visual Data Exploration

AbstractMobile devices are increasingly being used in the workplace. The combination of touch, pen, and speech interaction with mobile devices is considered particularly promising for a more natural experience. However, we do not yet know how everyday work with multimodal data visualizations on a mobile device differs from working in the standard WIMP workplace setup. To address this gap, we created a visualization system for social scientists, with a WIMP interface for desktop PCs, and a multimodal interface for tablets. The system provides visualizations to explore spatio‐temporal data with consistent WIMP and multimodal interaction techniques. To investigate how the different combinations of devices and interaction modalities affect the performance and experience of domain experts in a work setting, we conducted an experiment with 16 social scientists where they carried out a series of tasks with both interfaces. Participants were significantly faster and slightly more accurate on the WIMP interface. They solved the tasks with different strategies according to the interaction modalities available. The pen was the most used and appreciated input modality. Most participants preferred the multimodal setup and could imagine using it at work. We present our findings, together with their implications for the interaction design of data visualizations.

On the utility of 3D hand cursors to explore medical volume datasets with a touchless interface

Analyzing medical volume datasets requires interactive visualization so that users can extract anatomo-physiological information in real-time. Conventional volume rendering systems rely on 2D input devices, such as mice and keyboards, which are known to hamper 3D analysis as users often struggle to obtain the desired orientation that is only achieved after several attempts. In this paper, we address which 3D analysis tools are better performed with 3D hand cursors operating on a touchless interface comparatively to a 2D input devices running on a conventional WIMP interface. The main goals of this paper are to explore the capabilities of (simple) hand gestures to facilitate sterile manipulation of 3D medical data on a touchless interface, without resorting on wearables, and to evaluate the surgical feasibility of the proposed interface next to senior surgeons (N=5) and interns (N=2). To this end, we developed a touchless interface controlled via hand gestures and body postures to rapidly rotate and position medical volume images in three-dimensions, where each hand acts as an interactive 3D cursor. User studies were conducted with laypeople, while informal evaluation sessions were carried with senior surgeons, radiologists and professional biomedical engineers. Results demonstrate its usability as the proposed touchless interface improves spatial awareness and a more fluent interaction with the 3D volume than with traditional 2D input devices, as it requires lesser number of attempts to achieve the desired orientation by avoiding the composition of several cumulative rotations, which is typically necessary in WIMP interfaces. However, tasks requiring precision such as clipping plane visualization and tagging are best performed with mouse-based systems due to noise, incorrect gestures detection and problems in skeleton tracking that need to be addressed before tests in real medical environments might be performed.

Optimistic Programming of Touch Interaction

Touch-sensitive surfaces have become a predominant input medium for computing devices. In particular, multitouch capability of these devices has given rise to developing rich interaction vocabularies for “real” direct manipulation of user interfaces. However, the richness and flexibility of touch interaction often comes with significant complexity for programming these behaviors. Particularly, finger touches, though intuitive, are imprecise and lead to ambiguity. Touch input often involves coordinated movements of multiple fingers as opposed to the single pointer of a traditional WIMP interface. It is challenging in not only detecting the intended motion carried out by these fingers but also in determining the target objects being manipulated due to multiple focus points. Currently, developers often need to build touch behaviors by dealing with raw touch events that is effort consuming and error-prone. In this article, we present Touch, a tool that allows developers to easily specify their desired touch behaviors by demonstrating them live on a touch-sensitive device or selecting them from a list of common behaviors. Developers can then integrate these touch behaviors into their application as resources and via an API exposed by our runtime framework. The integrated tool support enables developers to think and program optimistically about how these touch interactions should behave, without worrying about underlying complexity and technical details in detecting target behaviors and invoking application logic. We discuss the design of several novel inference algorithms that underlie these tool supports and evaluate them against a multitouch dataset that we collected from end users. We also demonstrate the usefulness of our system via an example application.

VR–CAD integration: Multimodal immersive interaction and advanced haptic paradigms for implicit edition of CAD models

This paper presents an approach for the integration of Virtual Reality (VR) and Computer-Aided Design (CAD). Our general goal is to develop a VR–CAD framework making possible intuitive and direct 3D edition on CAD objects within Virtual Environments (VE). Such a framework can be applied to collaborative part design activities and to immersive project reviews. The cornerstone of our approach is a model that manages implicit editing of CAD objects. This model uses a naming technique of B-Rep components and a set of logical rules to provide straight access to the operators of Construction History Graphs (CHG). Another set of logical rules and the replay capacities of CHG make it possible to modify in real-time the parameters of these operators according to the user’s 3D interactions. A demonstrator of our model has been developed on the OpenCASCADE geometric kernel, but we explain how it can be applied to more standard CAD systems such as CATIA. We combined our VR–CAD framework with multimodal immersive interaction (using 6 DoF tracking, speech and gesture recognition systems) to gain direct and intuitive deformation of the objects’ shapes within a VE, thus avoiding explicit interactions with the CHG within a classical WIMP interface. In addition, we present several haptic paradigms specially conceptualized and evaluated to provide an accurate perception of B-Rep components and to help the user during his/her 3D interactions. Finally, we conclude on some issues for future researches in the field of VR–CAD integration.

Embedded human computer interaction

In this paper, human interaction with embedded or ubiquitous technology is considered. The techniques focus on the use of what might be termed “everyday” objects and actions as a means of controlling (or otherwise interacting with) technology. While this paper is not intended to be an exhaustive review, it does present a view of the immediate future of human–computer interaction (HCI) in which users move beyond the desktop to where interacting with technology becomes merged with other activity. At one level this places HCI in the context of other forms of personal and domestic technologies. At another level, this raises questions as to how people will interact with technologies of the future. Until now, HCI had often relied on people learning obscure command sets or learning to recognise words and objects on their computer screen. The most significant advance in HCI (the invention of the WIMP interface) is already some 40 years old. Thus, the future of HCI might be one in which people are encouraged (or at least allowed) to employ the skills that they have developed during their lives in order to interact with technology, rather than being forced to learn and perfect new skills.

Comparing In-Keyboard Pointing Devices: Considerations for the Practitioner

Pointing sticks and touchpads are the leading integrated pointing devices for portable computers. Yet research shows that users are frustrated with these devices. With WIMP interface firmly entrenched and mobile computer use drastically rising, attention must be paid to the virtues and limitations of integrated pointing devices. This article offers a mixture of qualitative and quantitative findings that clarify the issues and the design factors that affect performance and satisfaction with integrated pointing devices. Findings indicate that the rate-control pointing stick is a more efficient and preferred pointer to the position-control touchpad device, but that other design factors such as palm rest size, LCD technology differences, and device implementation differences might account for more variance in pointer effectiveness and user satisfaction than the device type itself. Contradictory findings in the recent literature are explained in terms of differences among these other factors. Practitioner guidelines for these technologies are also discussed.

Discourse structures in iconic communication

The success of the WIMP interface, the convergence towards multimedia platforms and the growing use of the Internet combine to place new demands on the systems we use to communicate with one another. Proposals for iconic languages are not new but the computer offers the ability to incorporate interactivity and animation into iconic writing and reading systems. One approach to developing computer-based iconic communication systems derives from the visual arts (i.e. graphic design, illustration, film animation, documentary video, etc.). An alternative approach is derived from computational linguistics in that the syntactic or semantic structure of language is retained while iconic images are used in place of words. Differences between the study of natural language and the design of an iconic language are discussed and the components of one particular system, IconText, are described. Of particular interest is the attempt to capture some pragmatic aspects of communication, both in terms of conveying the intentions of the author and concerning the sequential delivery of the message.

HyperArc: a task-oriented hypertext GIS interface

Abstract The rationale, development and -implementation of a task-oriented hypertext GIS interface (HyperArc) are described. HyperArc is a fully functional interface on an Apple Macintosh computer acting as a client to a host computer running the ARC/INFO GIS. The package has been developed using the HyperCard package, and makes full use of the Apple Macintosh WIMP interface, enabling the user to perform a range of standard ARC/INFO tasks, including a variety of map displays and data queries. The key advantage of this kind of interface is that it permits the creation of tasks or sets of spatial procedures which can be easily customised and executed by the end user without any knowledge of the system command language.

Wimps and nerds

Since the introduction of the Xerox Star computer and its more popular successor, the Apple Macintosh, the so-called WIMP interface has become predominant in software interfaces. In this paper we discuss the fact that the WIMP interface handles the physical, but not the cognitive interface to software. We propose to supplement the WIMP model of the physical interface with the NERD model of the cognitive interface. Like the WIMP model, the NERD model is designed to enhance interface consistency and to increase the power of the user while reducing cognitive load associated with the task.