Real-time Interactive Systems Research Articles

저가형 모션 캡처 장비를 이용한 실시간 상호작용 애니메이션 시스템

In this paper, we introduce a novel real-time, interactive animation system which uses real-time motion inputs from a low-cost motion-sensing device Kinect. Our system generates interaction motions between the user character and the counterpart character in real-time. While the motion of the user character is generated mimicking the user’s input motion, the other character’s motion is decided to react to the user avatar’s motion. During a pre-processing step, our system analyzes the reference motion data and generates mapping model in advance. At run-time, our system first generates initial poses of two characters and then modifies them so that it could provide plausible interacting behavior. Our experimental results show plausible interacting animations in that the user character performs a modified motion of user input and the counterpart character properly reacts against the user character. The proposed method will be useful for developing real-time interactive animation systems which provide a better immersive experience for users.

Using Blockchain based Authentication Solution for the Remote Surgery in Tactile Internet

Since the Tactile Internet has been considered as a new era of Internet, delivering real-time interactive systems as well as ultra-reliable and ultra-responsive network connectivity, tremendous efforts have been made to ensure authentication between communication’s parties to secure remote surgery. Since this human to machine interaction like remote surgery is critical and the communication between the surgeon and the tactile actor i.e. robot arms should be fully protected during the surgical procedure, a fully secure mutual user authentication scheme should be used in order to establish a secure session among the communicating parties. The existing methods usually require a server to ensure the authentication among the communicating parties, which makes the system vulnerable to single of point failure and not fit the design of such critical distributed environment i.e. tactile internet. To address these issues, we propose a new decentralized blockchain based authentication solution for tactile internet. In our proposed solution, there is no need for a trusted party; moreover, the decentralized nature of our proposed solution makes the authentication immutable, efficient, secure, and low latency requirement. The implementation of our proposed solution is deployed on Ethereum official test network Ropsten. The experimental results show that our solution is efficient, highly secured, and flexible.

Joint Computation Offloading, Power Allocation, and Channel Assignment for 5G-Enabled Traffic Management Systems

Due to the ever-increasing requirements of delay-sensitive and mission-critical applications in 5G, mobile edge computing is promising to react and support real-time interactive systems. However, it is still challenging to construct a 5G-enabled traffic management system, owing to the qualification of ultra-low latency and ubiquitous connectivity. Furthermore, the computing resources and storage capacities of edge nodes are limited, thus computation offloading is a fundamental issue for real-time traffic management. This paper puts forward a hybrid computation offloading framework for real-time traffic management in 5G networks. Specially, we consider both nonorthogonal-multiple-access-enabled and vehicle-to-vehicle-based traffic offloading. The investigated problem is formulated as a joint task distribution, subchannel assignment, and power allocation problem, with the objective of maximizing the sum offloading rate. After that, we prove its NP-hardness and decompose it into three subproblems, which can be solved iteratively. Performance evaluations illustrate the effectiveness of our framework.

User authentication in a tactile internet based remote surgery environment: Security issues, challenges, and future research directions

Recently, the Tactile Internet (TI) becomes a new era of the Internet. The TI provides ultra-reliable, ultra-responsive and intelligent network connectivity for delivering real-time control and physical haptic experiences from a remote location. The TI provides a different feeling to human–machine interaction by implementing the real-time interactive systems. In this review article, we discuss a generalized authentication model which can be used to perform authentication procedure among different communicating parties in order to secure remote surgery in the TI environment. By using the proposed authentication model, an authentication protocol can be designed so that an authenticated surgeon can use the robot/robotic arms to perform the surgery securely as well as remotely. Since the application is very critical, the important instructions provided by the surgeon to the robot/robotic arms must not be leaked in between the communication during the surgical procedure. To deal with this emerging research area, a secure mutual user authentication mechanism should be provided between a remote surgeon and the robot/robotic arms so that they can communicate securely using the established session key among them. Further, several security issues and challenges for such kind of communication are also discussed in this article. Finally, we discuss few points that need to be considered as future research works that are related to authentication for securing remote surgery in the TI environment.

Gaze-based predictive user interfaces: Visualizing user intentions in the presence of uncertainty

Abstract Human eyes exhibit different characteristic patterns during different virtual interaction tasks such as moving a window, scrolling a piece of text, or maximizing an image. Human-computer studies literature contains examples of intelligent systems that can predict user’s task-related intentions and goals based on eye gaze behavior. However, these systems are generally evaluated in terms of prediction accuracy, and on previously collected offline interaction data. Little attention has been paid to creating real-time interactive systems using eye gaze and evaluating them in online use. We have five main contributions that address this gap from a variety of aspects. First, we present the first line of work that uses real-time feedback generated by a gaze-based probabilistic task prediction model to build an adaptive real-time visualization system. Our system is able to dynamically provide adaptive interventions that are informed by real-time user behavior data. Second, we propose two novel adaptive visualization approaches that take into account the presence of uncertainty in the outputs of prediction models. Third, we offer a personalization method to suggest which approach will be more suitable for each user in terms of system performance (measured in terms of prediction accuracy). Personalization boosts system performance and provides users with the more optimal visualization approach (measured in terms of usability and perceived task load). Fourth, by means of a thorough usability study, we quantify the effects of the proposed visualization approaches and prediction errors on natural user behavior and the performance of the underlying prediction systems. Finally, this paper also demonstrates that our previously-published gaze-based task prediction system, which was assessed as successful in an offline test scenario, can also be successfully utilized in realistic online usage scenarios.

Generating Time: Rhythmic Perception, Prediction and Production with Recurrent Neural Networks

In the quest for a convincing musical agent that performs in real time alongside human performers, the issues surrounding expressively timed rhythm must be addressed. Current beat tracking methods are not sufficient to follow rhythms automatically when dealing with varying tempo and expressive timing. In the generation of rhythm, some existing interactive systems ignore the pulse entirely, or fix a tempo after some time spent listening to input. Since music unfolds in time, we take the view that musical timing needs to be at the core of a music generation system. Our research explores a connectionist machine learning approach to expressive rhythm generation, based on cognitive and neurological models. Two neural network models are combined within one integrated system. A Gradient Frequency Neural Network (GFNN) models the perception of periodicities by resonating nonlinearly with the musical input, creating a hierarchy of strong and weak oscillations that relate to the metrical structure. A Long Short-term Memory Recurrent Neural Network (LSTM) models longer-term temporal relations based on the GFNN output. The output of the system is a prediction of when in time the next rhythmic event is likely to occur. These predictions can be used to produce new rhythms, forming a generative model. We have trained the system on a dataset of expressively performed piano solos and evaluated its ability to accurately predict rhythmic events. Based on the encouraging results, we conclude that the GFNN-LSTM model has great potential to add the ability to follow and generate expressive rhythmic structures to real-time interactive systems

Model-based testing for building reliable realtime interactive music systems

The role of an Interactive Music System (IMS) is to accompany musicians during live performances, acting like a real musician. It must react in realtime to audio signals from musicians, according to a timed high-level requirement called mixed score, written in a domain specific language. Such goals imply strong requirements of temporal reliability and robustness to unforeseen errors in input, yet not much addressed by the computer music community.We present the application of Model-Based Testing techniques and tools to a state-of-the-art IMS, including in particular: offline and on-the-fly approaches for the generation of relevant input data for testing (including timing values), with coverage criteria, the computation of the corresponding expected output, according to the semantics of a given mixed score, the black-box execution of the test data on the System Under Test and the production of a verdict. Our method is based on formal models in a dedicated intermediate representation, compiled directly from mixed scores (high-level requirements), and either passed, to the model-checker Uppaal (after conversion to Timed Automata) in the offline approach, or executed by a virtual machine in the online approach. Our fully automatic framework has been applied to real mixed scores used in concerts and the results obtained have permitted to identify bugs in the target IMS.

The Effects of Interpersonal Attitude of a Group of Agents on User’s Presence and Proxemics Behavior

In the everyday world people form small conversing groups where social interaction takes place, and much of the social behavior takes place through managing interpersonal space (i.e., proxemics) and group formation, signaling their attentio to others (i.e., through gaze behavior), and expressing certain attitudes, for example, friendliness, by smiling, getting close through increased engagement and intimacy, and welcoming newcomers. Many real-time interactive systems feature virtual anthropomorphic characters in order to simulate conversing groups and add plausibility and believability to the simulated environments. However, only a few have dealt with autonomous behavior generation, and in those cases, the agents’ exhibited behavior should be evaluated by users in terms of appropriateness, believability, and conveyed meaning (e.g., attitudes). In this article we present an integrated intelligent interactive system for generating believable nonverbal behavior exhibited by virtual agents in small simulated group conversations. The produced behavior supports group formation management and the expression of interpersonal attitudes (friendly vs. unfriendly) both among the agents in the group (i.e., in-group attitude) and towards an approaching user in an avatar-based interaction (out-group attitude). A user study investigating the effects of these attitudes on users’ social presence evaluation and proxemics behavior (with their avatar) in a three-dimensional virtual city environment is presented. We divided the study into two trials according to the task assigned to users, that is, joining a conversing group and reaching a target destination behind the group. Results showed that the out-group attitude had a major impact on social presence evaluations in both trials, whereby friendly groups were perceived as more socially rich. The user’s proxemics behavior depended on both out-group and in-group attitudes expressed by the agents. Implications of these results for the design and implementation of similar intelligent interactive systems for the autonomous generation of agents’ multimodal behavior are briefly discussed.

Adaptive Micro-Channel Real-Time Interactive Systems of Nanchang Road Congestion Based on Improved Dijkstra Algorithm

The key technology research and development projects of national vehiclenetworking system needs, software products and components, developed withindependent intellectual property rights and core technologies based onimproved Dijkstra algorithm Nanchang road congestion adaptive system based onmicro-channel real-time interaction; main contents of transportation forNanchang adaptive significance of congestion vehicle networking applications,the status quo and research problems. GPS has immediate feedback andmicro-channel platform for mutual exchange of powerful features. Thuscountermeasures to enhance vehicle networking applications, Jiangxi Science andTechnology Park level. Developed based on the GPSNanchang traffic congestion adaptive micro-channel platform system, thesoftware product has low prices, professional, practical, easy installation,simple operation, with independent intellectual property rights, perfectlysuited for automotive units practical operation. Has a strong competitiveadvantage. The research results can be widely used in the province actual carservice units (including the use of e-commerce), a system price of Wu Yuan,automotive service products can be formed in Jiangxi Province, the developmentof new industries and generate significant economic benefits.

The Development of Computer Music Programming Systems

This article traces the history and evolution of Music Programming, from the early off-line synthesis programs of the MUSIC N family to modern realtime interactive systems. It explores the main design characteristics of these systems and their impact on Computer Music. In chronological fashion, the article will examine, with code examples, the development of the early systems into the most common modern languages currently in use. In particular, we will focus on Csound, highlighting its main internal aspects and its applications. The text will also explore the various paradigms that have oriented the design and use of music programming systems. This discussion is completed by a consideration of computer music eco-systems and their pervasiveness in today’s practice.

Remote FPGA Lab for Enhancing Learning of Digital Systems

Learning in digital systems can be enhanced through applying a learn-by-doing approach on practical hardware systems and by using Web-based technology to visualize and animate hardware behavior. The authors have reported the Web-based Remote FPGA Lab (RFL) which provides a novel, real-time control and visualization interface to a remote, always-on FPGA hardware implementation. The RFL helps students to understand and reason about digital systems operation, using interactive animation of signal behavior in an executing digital logic system, at any level of the design hierarchy. The RFL supports the creation of real-time interactive digital systems teaching demos. The article presents student RFL usage data and survey data which highlight improved student engagement, learning and achievement. The article describes the RFL architecture, communication interface, Web page functionality, user access administration and database management. The article also describes the RFLGen program, developed to automate user design integration into the Xilinx ISE VHDL-based RFL project wrapper for creation of FPGA configuration bitstreams and RFL animations.

Psychic systems and metaphysical machines: experiencing behavioural prediction with neural networks

We are living in a time of meta-organics and post-biology, where we perceive everything in our world as customizable and changeable. Modelling biology within a technological context allows us to investigate GEO-volutionary alternatives/alterations to our original natural systems, where augmentation and transmutation become standards in search of overall betterment (Genetically Engineered Organics). Our expectations for technology exceeds ubiquitous access and functional perfection and enters the world of technoetics, where our present hyper-functional, immersively multi-apped, borderline-prosthetic, global village devices fail to satiate our desires to be synaptically surpassed. Most real-time interactive behavioural systems in art utilize two and three dimensions. Artificial neural networks and prediction (Latin: from pr- before plus dicere to say) systems give rise to experimentation within the third-plus-one (3+1) dimension of the spacetime continuum. Spacetime in accordance with string theory requires the where and when to describe something in existence, rather than mere points in space. It explains the workings of the universe from both super-galactic and subatomic levels. Physical cosmology studies the motions of the celestial bodies in relation to the first cause (ie. primum movens) or the source of all-being. In book 12 of Metaphysics, Aristotle spoke of something which moves without (itself) being moved (by anything), and referred to being as motion. In quantum mechanics, a particle is described by a wave. Physiognomy is an evolving area of psychological research that describes a direct correlation between human gesture (or movement) and thought. Early twentieth-century Persian philosopher Abdul-Baha mentioned that the reality of man is his thought, not his material body. Prediction systems apply models of the mind, or artificial neural networks and machine learning strategies, to use the recorded past with the present to anticipate the future. This article investigates existing human and artificial psychic systems in hopes of identifying tools necessary in building a gesture prediction system. The relationship of intent to gesture and whether intent can be perceived without the evidence of gesture are two factors that must be tested.

A word from the editor

Welcome to the second issue of ACM Computers in Entertainment magazine! We continue the theme of Educating Children Through Entertainment with a diversity of topics on virtual reality, interactive theaters, storyrooms, playsets, and music. To get started, the Interviews section features video interviews with our distinguished advisory board members Quincy Jones and Seymour Papert. Quincy talks about education as fun learning, the impact of music on children, how technology affects the way music is created and produced, and digital distribution of music over the Internet. Seymour speaks of learning-rich entertainment, Logo and computer programming for children, technology in schools and education, and the future of home schooling and learning environments. In the Virtual Reality and Interactive Theaters section, Maria Roussou presents multisensory and immersive virtual reality worlds created for children, with particular attention given to the role and nature of interactivity in relation to learning, play, and narrative. Ute Ritterfeld et al. examine the short- and long-term impacts of entertainment education on children by experimenting with an interactive show in theaters intended to enhance childrens understanding of science. In the Storyrooms and Playsets section, Jaime Montemayor et al. discuss their research on physical interactive storytelling environments for children ages 5 to 6. Children are encouraged to create their own physical storytelling interactions. Tammie Hutto Egloff details a case study of interactive CD-ROM playsets with an analysis of age and gender differences, ergonomics, metaphor understanding, and learnability. One surprising result, according to her collected data, is that 75% of males and 77.8% of females (ages 3 to 5) preferred the kitchen playset to the workshop toys. In the Music section, Francois Pachet et al. describe the Continuator project that uses a reflective interactive system to trigger musical interest in children and create a stimulating, unsupervised music learning environment. Homei Miyashita portrays a collection of karaoke animation programs on television and the Internet to help children learn Japaneses traditional nursery rhymes. In the Synopsis Books et al. section, Kim Rose introduces Squeak as a language, a tool, and a media-authoring environment for children. Her book Powerful Ideas in the Classroom: Using Squeak to Enhance Math and Science Learning offers a Squeak-based curriculum using Etoy computer projects and off-computer activities to amplify learning. These and many more interesting topics make ACM Computers in Entertainment an exciting magazine! In the next issue, we will show the video interviews of our distinguished advisory board members Leonard Kleinrock, a father of the Internet who created the basic principles of packet switching, and Richard Edlund, a four-time Academy Award® winner for visual effects work on Star Wars and "Raiders of the Lost Ark." Since I first approached ACM with the idea of this magazine in May 2002, I have received overwhelming support from my friends and colleagues in the entertainment industry, academia, computer and consumer electronics companies, and R&D centers around the world. I want to express my sincere gratitude to all my editorial board advisors who have helped make this magazine a reality. I would also like to take this opportunity to congratulate some of them who have made headline news recently: 1. Dr. Alan Kay, Senior HP Fellow and President of Viewpoints Research Institute, won the engineering professions highest honors for 2004, presented by the National Academy of Engineering. Alan shared the $500,000 Charles Stark Draper Prize for his vision, conception, and development of the worlds first practical networked personal computers. Alan was one of our featured interviewees last year in the inaugural issue of this magazine. 2. Scott Watson, Senior VP and CTO of Buena Vista Datacastings MovieBeam, won the Consumer Electronics Show (CES) Innovations 2004 Design and Engineering Showcase Honoree Award for MovieBeam. The award honors superior design and engineering among the years most technologically advanced products in a wide range of consumer electronics categories. 3. Professor Elaine Chew at USC received the 2004 Early Career Award from the National Science Foundation and a five-year, $500,000 grant for research, teaching, and outreach activities in the development of computational music perception and cognition models for real-time interactive music systems. 4. Professor Cynthia Breazeal at the MIT Media Lab was selected a finalist for the 2003 National Design Award in Communication and a speaker at the 2004 National Academy of Engineerings Frontiers of Engineering Symposium. We will be showing a video interview with Cynthia in an upcoming issue of this magazine. Enough said -- please enjoy the magazine and send me your comments and feedback. Thank you very much for your continuing support. Sincerely, Newton Lee Editor-in-Chief Los Angeles March 2004

A Working Model for Real-time Interactive Distance Education Support Systems

In Japan, most of the real time distance learning systems constructed over JGN (Japan Gigabit Network) are simple applications of video conference system equipped with high-quality audio/video transmission equipment. Such systems are not modeled to add distance educational support such as real time floor control, and are insufficient and inefficient to perform real time distance lecture or seminar when the number of sites get larger. This paper proposes a Working Model for distributed computer systems supporting real-time interactive distance lectures or seminars over JGN. As an application of the working model, functions for floor control and tele-presentation were discussed. The utility of the proposed working model was confirmed with an experimental system.

The Evolution of evaluation: Lessons from the Message Understanding Conferences

The Message Understanding Conferences (MUCs) represent one of the earliest and longest running efforts to evaluate language understanding technology. This article reviews the history of the MUCs and their evolution towards the use of common training and blind test sets, automated scoring, task decomposition into modular building blocks and tools for portability across languages and applications. Now that evaluation has become an accepted part of the developer's toolkit, it is important to understand the interplay between evaluation methods and the state of research. MUC was successful in generating excitement about text processing problems and in attracting talented researchers to the area. It also provided a functional decomposition of the information extraction problem into a series of simpler problems, thus allowing researchers to demonstrate successful systems and to spin off commercial products. However, the ultimate goal of accurate information extraction has been elusive; systems have become faster and cheaper to build, the evaluations have become harder, but overall accuracy in information extraction has improved only modestly. The MUC experience contrasts with experiences in other evaluations. For example, the spoken evaluation in the Air Travel Information System (ATIS) has shown dramatic improvement in error rate over time, but those evaluations were limited to a single domain and the metrics did not address interaction, even though real-time interactive systems were available. Looking across the history of MUC in the context of related evaluations, we can draw important lessons about the need for evaluation to evolve with the technology it evaluates, to balance costs against benefits and to weigh the divergent needs of the multiple stake-holders— developers, funders and users—in order to provide continuity while also providing the next set of challenges to the research community.

Distributed Database Designs and Computation Strategies for Networked Interactive Stimulations

In real-time interactive training and simulation systems, changes of the natural terrain surface and other objects are involved. Distributed geographical database and visual simulators are being built to meet the requirements of low cost and high efficiency. In this paper, four approaches for distributing terrain databases and simulation entities across loosely coupled networks of simulators are examined. An analytical model is developed, which can be used to quantitatively estimate the performance of a distributed system. An experimental simulation study is also conducted to validate the analytical method.

Strategies of Real-time Cartography

As the application of interactive computer graphic methods to cartographic tasks continues to develop and mature, it is useful and instructive to examine these developments in a deeper conceptual context. Possibilities for cartographic communication have been greatly expanded with such interactive cartographic computer systems. A key concept in the power and flexibility of such interactive systems is that of real and virtual maps and transformations that can be made between them. One can now look past current interactive cartographic systems for display, analysis and map production to the potential of more powerful and flexible real-time interactive cartographic systems which can deal with more complex forms of cartographic situations and include the manipulation of a number of aspects of spatiotemporal dynamics. These include cartographic object manipulation and surface exploration, the display of cartographic data as it changes through time, and combinations of the above.

Computer modelling and control of manufacturing systems with particular reference to the tailoring industry

There is considerable potential for computers integrated in the manufacturing process at factory and shop-floor level. In particular, owing to their current low price, minicomputers are attractive for use in realtime interactive computer-aided control systems where the supervisory staff are a significant part of the control system. Computers tend to have a low-level of application in a systems engineering sense, i.e. they are typically used for stock control, cataloguing and data processing, which are only elements of a control system. There is much to be done in high-level dynamic feedforward control to help management by incorporating forecasting techniques with manufacturing-system models and assessments of current system state in a dynamic and interactive manner. If this potential can be realised by incorporating computer hardware, visual data-display terminals and information collection devices into the man/machine manufacturing process, then management will be more able to carry out their roles. Complex automation of the factory is not being proposed. On the contrary, it is believed that an appreciation of the computer-system limitations, coupled with the need to exploit modern control theory and computing techniques, will lead to systems in which the key controlling role of the supervisors will be strengthened, becoming dominant and more satisfying. The computer will handle the details of information searching and processing. The man will be the controller. This paper summarises and clarifies techniques by which higher forms of control might be realised.