Direct Manipulation Of Objects Research Articles

Bidirectional Object-Oriented Programming: Towards Programmatic and Direct Manipulation of Objects

Many bidirectional programming languages, which are mainly functional and relational, have been designed to support writing programs that run in both forward and backward directions. Nevertheless, there is little study on the bidirectionalization of object-oriented languages that are more popular in practice. This paper presents the first bidirectional object-oriented language that supports programmatic and direct manipulation of objects. Specifically, we carefully extend a core object-oriented language, which has a standard forward evaluation semantics, with backward updating semantics for class inheritance hierarchies and references. We formally prove that the bidirectional evaluation semantics satisfies the round-tripping properties if the output is altered consistently. To validate the utility of our approach, we have developed a tool called BiOOP for generating HTML documents through bidirectional GUI design. We evaluate the expressiveness and effectiveness of BiOOP for HTML webpage development by reproducing ten classic object-oriented applications from a Java Swing tutorial and one large project from GitHub. The experimental results show the response time of direct manipulation programming on object-oriented programs that produce HTML webpages is acceptable for developers.

Trade-Off between Task Accuracy, Task Completion Time and Naturalness for Direct Object Manipulation in Virtual Reality

Virtual reality devices are used for several application domains, such as medicine, entertainment, marketing and training. A handheld controller is the common interaction method for direct object manipulation in virtual reality environments. Using hands would be a straightforward way to directly manipulate objects in the virtual environment if hand-tracking technology were reliable enough. In recent comparison studies, hand-based systems compared unfavorably against the handheld controllers in task completion times and accuracy. In our controlled study, we compare these two interaction techniques with a new hybrid interaction technique which combines the controller tracking with hand gestures for a rigid object manipulation task. The results demonstrate that the hybrid interaction technique is the most preferred because it is intuitive, easy to use, fast, reliable and it provides haptic feedback resembling the real-world object grab. This suggests that there is a trade-off between naturalness, task accuracy and task completion time when using these direct manipulation interaction techniques, and participants prefer to use interaction techniques that provide a balance between these three factors.

Time Enhanced Architectural Modelling (T.E.A.M.): Virtual reality project for the planning and visualization of kinetic architecture and dynamic design

T.E.A.M. is a research project that was created to facilitate the design of kinetic projects and components created through a computational design process. Time is the ingredient that allows dynamism. The following paper examines the core features of Platform One, an experimental digital application born within the project and aimed at supporting designers during the development, in a VE (Virtual Environment) of dynamic architectures and components, as well as the principles that inspired it. The application presents two key features: the first one is that everything modified and developed in the VE retains its geometric characteristics, allowing the user to reach an informed 3D model at the end of the process; the second one is the ease and enjoyment with which the user manipulates complex dynamic geometries in the three-dimensional environment through a natural interface design approach that focuses on direct manipulation of architectural objects and components. The simulator is designed to be used in a 6DOF virtual environment using a commercial VR headset. It has currently been loaded with several archetypal test architectures and soon it will be available to designers who want to test their work with it.

Human-Inspired Haptic-Enabled Learning From Prehensile Move Demonstrations

Research on robotic manipulation of fragile, compliant objects, such as food items, is gaining traction due to its game-changing potential within the food production and retailing sectors, currently characterized by manually-intensive and highly repetitive tasks. Food products exhibit high levels of frailness, biological variation, and complex 3D shapes and textures. For these reasons, introducing greater levels of robotic automation in the food and agricultural sectors remains an important challenge. This paper addresses this challenge by developing a human-centred, haptic-based, Learning from Demonstration (LfD) policy that enables pre-trained autonomous grasping of food items using an anthropomorphic robotic system. The policy combines data from teleoperation and direct human manipulation of objects, embodying human intent and interaction areas of significance. We evaluated the proposed solution against a recent state-of-the-art LfD policy as well as against two standard impedance controller techniques. Results show that the proposed policy performs significantly better than the other considered techniques, leading to high grasping success rates while guaranteeing the integrity of the food at hand.

AR Object Manipulation on Depth-Sensing Handheld Devices

Recently released, depth-sensing-capable, and moderately priced handheld devices support the implementation of augmented reality (AR) applications without the requirement of tracking visually distinct markers. This relaxed constraint allows for applications with significantly increased augmentation space dimension, virtual object size, and user movement freedom. Being relatively new, there is currently a lack of study on issues concerning direct virtual object manipulation for AR applications on these devices. This paper presents the results from a survey of the existing object manipulation methods designed for traditional handheld devices and identifies potentially viable ones for newer, depth-sensing-capable devices. The paper then describes the following: a test suite that implements the identified methods, test cases designed specifically for the characteristics offered by the new devices, the user testing process, and the corresponding results. Based on the study, this paper concludes that AR applications on newer, depth-sensing-capable handheld devices should manipulate small-scale virtual objects by mapping directly to device movements and large-scale virtual objects by supporting separate translation and rotation modes. Our work and results are the first step in better understanding the requirements to support direct virtual object manipulation for AR applications running on a new generation of depth-sensing-capable handheld devices.

A Tangible System for Learning Relational Algebra

Tangible User Interfaces (TUIs) are those in which users interact with a digital system through the direct manipulation of physical objects (tokens). Tokens are directly linked to a certain data/functionality within the system, so manipulation of these objects affects the system behavior. This paper introduces TanQuery, a tangible system to support the process of learning Relational Algebra. TanQuery incorporates components to detect and track tokens, and to analyze and execute query trees. The system was tested by university students, and obtained results allowed to observe and analyze that students found this type of interface useful and pleasant.
 Keywords: Tangible user interfaces; relational algebra.

Dressing disrupted: negotiating care through the materiality of dress in the context of dementia

This paper explores how the materiality of dress mediates and shapes practices of care in the context of dementia. Earlier research called for an approach to conceptualising care that recognised the role played by everyday artefacts. We extend this to a consideration of dress and dressing the body in relation to people with dementia that involves the direct manipulation of material objects, as well as the materiality of bodies. The paper draws on an ESRC funded study Dementia and Dress, which examined experiences of dress for people with dementia, families and care‐workers using ethnographic and qualitative methods. Our analysis explores the process of dressing the body, the physicality of guiding and manipulating bodies into clothing, dealing with fabrics and bodies which ‘act back’ and are resistant to the process of dressing. We consider how the materiality of clothing can constrain or enable practices of care, exploring tensions between garments that support ease of dressing and those that sustain identity. Examining negotiations around dress also reveals tensions between competing ‘logics’ of care (Mol 2008).

Enhancing Engineering Students' Performance on Design Task: The Box of Parts

Most definitions of engineering give machines and mechanical objects a central role. Engineers are makers and users of mechanical objects in their environment. Research supports the notion that interactions with engineered artifacts enhance engineering learning. This study introduces a task simulating a real-world engineering application and uses this task to examine how aptitudes, interests, and direct manipulation of mechanical objects influence performance. We hypothesized that engineering students would generate better assembly instructions when they had the box of component parts (BOP) than when they had the engineering drawing only. We also hypothesized that student's mechanical aptitude (MA) and interests in things each would interact with experimental condition's impact on performance. First-year engineering students (N = 383) created assembly instructions in a mixed experimental and correlational design. A random half was assigned to create instructions with a drawing only, whereas the other half created with both a drawing and a box of component parts present. Assembly instructions were evaluated by professional engineers blind to experimental conditions. They rated instructions from the BOP group as superior to those coming from the control group. Students with greater mechanical aptitude received better evaluations, but there was no evidence the experimental variable was moderated either by mechanical aptitude or by thing orientation (TO). This study suggests that mechanical objects can enhance engineering instruction, especially when they are aligned with professional practice.

A tablet computer application for conceptual design

In the conceptual design phase, solutions are reached through an iterative, high-paced and often chaotic manner. Conventional advanced structural analysis software is often too advanced and insufficiently agile to follow this high-paced work pattern. Premature use of advanced structural analysis tools can negatively affect conceptual understanding and the quality of the conceptual design. The multi-touch interfaces of today's tablet computers give the user a strong feeling of direct manipulation of objects on the screen. This is interesting for structural mechanics applications, enabling direct manipulation of the structural model on the screen so as to have a better understanding and feeling of the structural behaviour. A tablet computer application for the conceptual design phase, which uses this type of direct manipulation interface, has been developed.

The visuospatial functions in children after cerebellar low-grade astrocytoma surgery: A contribution to the pediatric neuropsychology of the cerebellum.

The aim of this study was to specify whether cerebellar lesions cause visuospatial impairments in children. The study sample consisted of 40 children with low-grade cerebellar astrocytoma, who underwent surgical treatment and 40 healthy controls matched with regard to age and sex. Visuospatial abilities were tested using the spatial WISC-R subtests (Block Design and Object Assembly), Rey-Osterrieth Complex Figure, Benton Judgment of Line Orientation Test, PEBL Mental Rotation Task, and Benton Visual Retention Test. To exclude general diffuse intellectual dysfunction, the WISC-R Verbal Intelligence IQ, Performance IQ, and Full-Scale IQ scores were analysed. Post-surgical medical consequences were measured with the International Cooperative Ataxia Rating Scale. Compared to controls, the cerebellar group manifested problems with mental rotation of objects, visuospatial organization, planning, and spatial construction processes which could not be explained by medical complications or general intellectual retardation. The intensity of visuospatial syndrome highly depends on cerebellar lesion side. Patients with left-sided cerebellar lesions display more severe spatial problems than those with right-sided cerebellar lesions. In conclusion, focal cerebellar lesions in children affect their visuospatial ability. The impairments profile is characterized by deficits in complex spatial processes such as visuospatial organization and mental rotation, requiring reconstruction of visual stimuli using the imagination, while elementary sensory analysis and perception as well as spatial processes requiring direct manipulation of objects are relatively better preserved. This pattern is analogous to the one previously observed in adult population and appears to be typical for cerebellar pathology in general, regardless of age.

Tactile reasoning: hands-on versus hands-off—What is the difference?

This study investigates tactile reasoning in relation to sense-making in the context of designing a new non-traditional information environment. We define tactile reasoning as an interaction technique that supports analytical reasoning by the direct manipulation of information objects in the graphical user interface. When people directly manipulate data, for example, by moving individual pieces of information to create temporary groups or sequences or by eliminating pieces of information from a group, we hypothesize that this can enhance their sense-making and analytical reasoning ability by helping them discover new explanatory relationships created by the rearranged pieces of information. Our study used a card-sorting task where participants were either allowed or not allowed to touch and manipulate the cards to look for information or to construct groups of information that could provide meaningful explanations. Participants were asked to think aloud as they constructed an understanding of the information contained in the cards. Results showed that being able to manipulate the cards has a positive effect on analytical reasoning performance. In the more difficult task conditions, participants who were allowed to use their hands were observed to make 99 sets of sense-making strategies, in comparison with the 50 by participants who were not allowed to use their hands.

Electromagnetic Linear Microdrive for Braille Screen: Control and Circuit Test

Graphical interfaces based on visual representation and direct manipulation of objects make the adequate use of computers quite difficult for people with reduced sight. A new type of graphical Braille screen is developed. Permanent magnet linear actuator intended for driving a needle in Braille screen has been optimized. The recently developed permanent magnet linear electromagnetic actuator for driving a needle in a Braille screen and the circuit testing method using micro r obots are discussed. Also, a planar and spatial motion tactile display with incorporated electromagnetic linear drives into its links is designed. The optimization is carried out with respect to minimal magnet motive force ensuring required minimum electromagnetic force on the mover. The optimization factors include dimensions of the cores and movable parts under additional constraint for overall dimension of the actuator. Finite element analysis, response surface methodology and design of experiments have been employed for the optimization. The static force characteristics and magnetic field distribution is studied when varying the parameters. 

Cross-cultural comparison of hand gestures of Japanese and Germans for tabletop systems

Goal was to study cultural differences between Japanese and Germans gesture use when interacting with a map and a video walkthrough application for a table top system. Japanese and Germans’ choice of gestures was compared in a quasi-experimental design. Gestures had to be generated for two different scenarios, an electronic map and a video walkthrough. Data revealed that physical aspects of hand gestures such as hand shape, focus on motion pattern, and preference of simple, one-hand gestures are similar between Japanese and Germans. However, Japanese and Germans differed in the choice of symbolic and metaphorical gestures and in the frame of reference and perspective taken for performing gestures. Furthermore, differences between Germans and Japanese were larger for the map application than for the video walkthrough. Culture partly affects the choice of hand gestures for table top systems. Designers of gesture vocabularies for tabletop systems have to carefully consider which referents are carried out by hand gestures alone. Gesture input appears to be appropriate for the direct manipulation of objects or real world applications. However, abstract functions or artificial realities should not be handled by gestures alone. Also, designers should consider the development of culture-centered interfaces.

Haptic Feedback in Teleoperation in Micro- and Nanoworlds

Robotic systems have been developed to handle very small objects, but their use remains complex and necessitates long-duration training. Simulators, such as molecular simulators, can provide access to large amounts of raw data, but only highly trained users can interpret the results of such systems. Haptic feedback in teleoperation, which provides force feedback to an operator, appears to be a promising solution for interaction with such systems, as it allows intuitiveness and flexibility. However, several issues arise while implementing teleoperation schemes at the micro- and nanoscale, owing to complex force fields that must be transmitted to users and scaling differences between the haptic device and the manipulated objects. Major advances in such technology have been made in recent years. In this chapter, we review the main systems in this area and highlight how some fundamental issues in teleoperation for micro- and nanoscale applications have been addressed. We consider three types of teleoperation, including (a) direct (manipulation of real objects), (b) virtual (use of simulators), and (c) augmented (combining real robotic systems and simulators). Remaining issues that must be addressed for further advances in teleoperation for micro- and nanoworlds are also discussed, including (a) comprehension of phenomena that dictate very small object (<500 micrometers) behavior and (b) design of intuitive 3-D manipulation systems. Design guidelines to realize an intuitive haptic feedback teleoperation system at the micro- and nanoscale level are proposed.

''A Pierced Bottle leaks not!?'' the Heuristic-branched out e-model of animated ''Science School'' in Year 4

Animated learning types have proven their superiority over the traditional forms of learning. Do we know of any child who does not like to watch animated films? For such content one needs not to be further encouraged! In the same light, it is a rewarding form of learning in primary grades. If we upgrade this form of learning with direct manipulation of animated objects and we present it in the context of heuristic branched out e-model of learning, we can extend its educational purposes and its foundation. The purpose of this paper is to present the heuristic branched out e-model of learning applied to animated films with direct manipulation of animated content. Our assumption is that the presented example of an animated scientific task ''Science School'' will also encourage and develop pupils' functional thinking skills, and as a model (approach) it is also applicable to all curriculum areas in the first grades of primary school.

Developing and Testing a Human-Based Gesture Vocabulary for Tabletop Systems

The goal was to study the natural and intuitive use of surface gestures for the development of a tabletop system. Furthermore, the effect of expertise on choice of gestures was examined. It is still not well understood what kinds of gestures novice users choose when they interact with gesture recognition systems. First, novices' and experts' choice of gestures for a tabletop system was compared in a quasiexperimental design. Second, memorability of novices' and experts' gesture sets derived from the first study was compared in an experimental study. Third, memorization of hand shape and motion path was examined in a further experiment. Data revealed user preferences for specific hand shapes and motion paths. Choice of gestures was affected by the size of the manipulated object,expertise, and nature of the command (direct manipulation of objects vs. assessment of abstract functions). Follow-up experiments revealed that the novices' gesture set was better memorized than were the experts' gesture set. Furthermore, the motion path of a gesture is better memorized than the specific hand shapes for a gesture. Expertise affects the choice of gesture to a certain degree. It is therefore essential to involve novice users in the development of gesture vocabularies. Gestures for technical systems should be simple and should involve distinctive motion patterns instead of focusing on specific hand shapes or number of fingers. Abstract or symbolic gestures should be avoided. Results of the study can be applied to the development of surface gestures for tabletop systems.

Natural Interaction Metaphors for Functional Validations of Virtual Car Models

Natural Interaction in virtual environments is a key requirement for the virtual validation of functional aspects in automotive product development processes. Natural Interaction is the metaphor people encounter in reality: the direct manipulation of objects by their hands. To enable this kind of Natural Interaction, we propose a pseudophysical metaphor that is both plausible enough to provide realistic interaction and robust enough to meet the needs of industrial applications. Our analysis of the most common types of objects in typical automotive scenarios guided the development of a set of refined grasping heuristics to support robust finger-based interaction of multiple hands and users. The objects' behavior in reaction to the users' finger motions is based on pseudophysical simulations, which also take various types of constrained objects into account. In dealing with real-world scenarios, we had to introduce the concept of Normal Proxies, which extend objects with appropriate normals for improved grasp detection and grasp stability. An expert review revealed that our interaction metaphors allow for an intuitive and reliable assessment of several functionalities of objects found in a car interior. Follow-up user studies showed that overall task performance and usability are similar for CAVE and HMD environments. For larger objects and more gross manipulation, using the CAVE without employing a virtual hand representation is preferred, but for more fine-grained manipulation and smaller objects, the HMD turns out to be beneficial.

Digital Games for Young Children Ages Three to Six: From Research to Design

Young children ages 3 to 6 play a wide range of digital games, which are now available on large screens, handheld screens, electronic learning systems, and electronic toys, and their time spent with games is growing. This article examines effects of digital games and how they could be designed to best serve children's needs. A small body of research has found that games—when well designed—can provide rich, fun, interactive experiences that can foster young children's learning, cognitive development, skill building, social interactions, physical activity, and healthy behaviors. Research on games that are not well designed has found that violent content can lead to fear, hostility, desensitization, and aggressive behavior; stereotyped portrayals of characters can lead to stereotyped beliefs about others and oneself; and game playing can consume valuable time that could have been better spent in exploratory play, direct manipulation of objects in the environment, physical activity, and social interaction. More research is needed to discover the potential strengths and drawbacks of games for this age group to improve game design and guide decision-making about game purchases and implementation. Since young children are especially vulnerable to media messages, it is important that their media match their developmental needs, interests, and abilities. New data collection methods are enabling researchers to investigate children's responses to digital games with greater depth and accuracy and to discover how those responses are associated with outcomes. For example, software can record the time spent playing a game, time spent on specific game challenges, errors made, use of help and remediation, amount of exposure to educational content, and areas of success in the game. Data can also be collected with systems that record eye movements, technologies that recognize emotional responses to games by recording subtle shifts in facial expression, and functional magnetic resonance imaging of brain activity during game play, all of which can contribute to our understanding of processes and effects of young children's game playing. The results of research using these new methods and more traditional methods will help to discover principles of game design that can help make games more developmentally appropriate and beneficial, building on the ways young children naturally play and learn.

Toward an integrated account of object and action selection: A computational analysis and empirical findings from reaching-to-grasp and tool-use

The act of reaching for and acting upon an object involves two forms of selection: selection of the object as a target, and selection of the action to be performed. While these two forms of selection are logically dissociable, and are evidently subserved by separable neural pathways, they must also be closely coordinated. We examine the nature of this coordination by developing and analyzing a computational model of object and action selection first proposed by Ward [Ward, R. (1999). Interactions between perception and action systems: a model for selective action. In G. W. Humphreys, J. Duncan, & A. Treisman (Eds.), Attention, Space and Action: Studies in Cognitive Neuroscience. Oxford: Oxford University Press]. An interesting tenet of this account, which we explore in detail, is that the interplay between object and action selection depends critically on top-down inputs representing the current task set or plan of action. A concrete manifestation of this, established through a series of simulations, is that the impact of distractor objects on reaching times can vary depending on the nature of the current action plan. In order to test the model's predictions in this regard, we conducted two experiments, one involving direct object manipulation, the other involving tool-use. In both experiments we observed the specific interaction between task set and distractor type predicted by the model. Our findings provide support for the computational model, and more broadly for an interactive account of object and action selection.

A novel asynchronous access method with binary interfaces.

BackgroundTraditionally synchronous access strategies require users to comply with one or more time constraints in order to communicate intent with a binary human-machine interface (e.g., mechanical, gestural or neural switches). Asynchronous access methods are preferable, but have not been used with binary interfaces in the control of devices that require more than two commands to be successfully operated.MethodsWe present the mathematical development and evaluation of a novel asynchronous access method that may be used to translate sporadic activations of binary interfaces into distinct outcomes for the control of devices requiring an arbitrary number of commands to be controlled. With this method, users are required to activate their interfaces only when the device under control behaves erroneously. Then, a recursive algorithm, incorporating contextual assumptions relevant to all possible outcomes, is used to obtain an informed estimate of user intention. We evaluate this method by simulating a control task requiring a series of target commands to be tracked by a model user.ResultsWhen compared to a random selection, the proposed asynchronous access method offers a significant reduction in the number of interface activations required from the user.ConclusionThis novel access method offers a variety of advantages over traditionally synchronous access strategies and may be adapted to a wide variety of contexts, with primary relevance to applications involving direct object manipulation.