Interactive Projection System Research Articles

Effectiveness of using digital interactive projection media systems in teaching vocational subjects and in professional activities

The article shows the technology's effectiveness and reveals the students' readiness to use digital interactive projection media systems in educational and professional activities. The content of the interactive education process and its components are disclosed. The essence and significance of digital interactive projection media systems in teaching professional subjects are shown. As an example of digital interactive projection systems, an effective interactive software-technological complex based on SMART Board is shown, the interactive functions of the interactive SMART Board software created specifically for the educational space in teaching vocational subjects and the advantages of the interactive panel in the educational space of a higher school are revealed. Ways of developing the teacher's professional competencies when using an interactive panel or blackboard in classes at a higher school are analyzed. The effectiveness of electronic textbooks, manuals electronic educational and methodological complexes, informative and reference multimedia publications, game educational and developmental publications, scientific research STEM centers, multimedia classes, virtual laboratories, virtual classes, etc. in teaching vocational subjects is shown.

Research on Chinese traditional culture development art innovation design and development system

Seeking the integration and innovation of traditional culture and modern technology, through the digital innovation and design research of traditional Chinese culture, the traditional Chinese ancient painting art and contemporary digital technology will go from parallel to active intersection and construct a new era from traditional Chinese culture: expressive form and symbol novelty. While realizing the creative concept design of contemporary Chinese traditional culture, the establishment of an infrared interactive projection system has been completed, and innovations have been made to meet the interactive projection requirements of traditional Chinese traditional culture for complicated technologies such as separation of pen touch points from background images and target tracking of pen touch points, with the breakthrough. They combined interactive projection technology with traditional Chinese culture for re-creation and re-expression, breaking through the inertial cognition of traditional Chinese culture with vision as the main body, emphasizing the correlation between hearing and traditional Chinese culture, and realizing conceptual innovation by designers and architectural designers. Technical personnel cooperate and communicate across borders and finally complete the multi-sensory new experience of digital art expression, providing a new way for digital innovation of traditional art.

An interactive holographic projection system that uses a hand-drawn interface with a consumer CPU

Holography is a promising technology for photo-realistic three-dimensional (3D) displays because of its ability to replay the light reflected from an object using a spatial light modulator (SLM). However, the enormous computational requirements for calculating computer-generated holograms (CGHs)—which are displayed on an SLM as a diffraction pattern—are a significant problem for practical uses (e.g., for interactive 3D displays for remote navigation systems). Here, we demonstrate an interactive 3D display system using electro-holography that can operate with a consumer’s CPU. The proposed system integrates an efficient and fast CGH computation algorithm for line-drawn 3D objects with inter-frame differencing, so that the trajectory of a line-drawn object that is handwritten on a drawing tablet can be played back interactively using only the CPU. In this system, we used an SLM with 1,920 times 1,080 pixels and a pixel pitch of 8 μm × 8 μm, a drawing tablet as an interface, and an Intel Core i9–9900K 3.60 GHz CPU. Numerical and optical experiments using a dataset of handwritten inputs show that the proposed system is capable of reproducing handwritten 3D images in real time with sufficient interactivity and image quality.

Creating different learning experiences: assessment of usability factors in an interactive three-dimensional holographic projection system for experiential learning

Innovative interfaces for the display and control of information constitute an essential topic for interactive experiential learning. In this study, an interactive three-dimensional (3D) holographic projection system was developed. This system was used in a physiology-based experiential learning experiment. Learners used noncontact somatosensory methods to manipulate 3D learning objects (targets) and learned the characteristics of physiological structures in a 3D holographic projection environment. The learners did not require a physical button interface. Embodied gesture recognition was implemented in this interactive system. Furthermore, this study explored the system’s usability factors to improve the human–computer interaction and availability of the system. A total of 60 participants (30 female and 30 male) participated in a usability experiment for this 3D interactive holographic projection learning system. The participants were required to complete an interactive experiential learning task concerning the physiological structures of human organs. At the end of the task, each participant was asked to complete a questionnaire featuring 5-point Likert scales. Four crucial system usability factors were proposed through principal component analysis. These factors included ‘labelling’, ‘continuity’, ‘backlash’, and ‘ambiences’. Gender had no significant effect on any of these factors (p > 0.05). Further, the learner’s experiential learning characteristics and human–computer interaction modality are described based on the results of the usability study.

The Finger-Based Interactive Projection using a Monocular Camera

Ordinary projection screen is not sensitive to interaction, it cannot meet the demands of teaching, virtual reality, and other applications. Due to the fact that people always use hands to complete a variety of human–computer interaction, the finger-based interactive projection technology is worth being researched. In this paper, an ordinary monocular camera is used to acquire video frame on projection screen, and the touch signal of finger in frame is used as the input of interactive projection system. Because the differences between spatial frequency of common digital camera and the projection screen is small, the frame obtained from camera will contain moire fringe, which needs to be filtered in image frequency domain. Then the difference between current frame edge and previous frame edge is calculated to obtain moving object edge clues. According to these clues, the most possible contour curve is searched in current frame edge, and the curve is fitted by polynomial approximation method. Its curvature integration is used to match with the curvature integration of finger template curve. After that the fingers in the curve are recognized. Because color information is not needed, this method can be used to recognize gloved fingers. Finally, finger shadow is used to judge whether the finger touches projection screen to complete interactive process. The experiments of writing and collaboratively rotating picture on projector screen show that this method can effectively complete interactive operation with the projection screen and can realize the multi-user operation.

Image Noise Preprocessing of Interactive Projection System Based on Switching Filtering Scheme

The interactive projection systems based on deep images are usually disturbed by the mixed noise. Generally, several filtering methods are used in combination to resolve this problem. Although the hybrid filter can guarantee the accuracy of the image, but the algorithm is complex and time‐consuming, which affects the real‐time performance of the interactive projection system. In this paper, the switching system method is introduced into the filter for the first time, and an arbitrary switching filter algorithm is proposed and applied to the depth image filtering system based on Kinect sensor. The experimental results demonstrate and validate that the proposed switching filter algorithm not only effectively removes the noise but also ensures the real‐time performance of tracking and achieves good target tracking performance, which makes it applicable in various image filtering processing systems.

FUTUREGYM: A gymnasium with interactive floor projection for children with special needs

Abstract Interpersonal interaction is one of the fundamental factors for successful inclusion in education for children with special needs, including children suffering from autism spectrum disorders (ASD) and/or intellectual disabilities (ID). In order to increase opportunities for interpersonal interactions among children, an interactive school gymnasium called FUTUREGYM, with a large-scale, interactive floor projection system in a school setting is proposed. As part of this study, high performance projectors (approximately 545-inch of total screen size) and tracking cameras mounted on the ceiling were installed in the gymnasium of a special needs school; these devices facilitated overlaying of individual visual aids for children with special needs. Visual aids help these children to be aware of social cues, a behavior that is considered important for the development of social interaction. Group activities to be conducted in FUTUREGYM were designed through careful consultations with teachers at the school: (1) Group running, called Circle-Run, provides an opportunity to experience a behavior that requires coordination with others, and (2) A group exergame, called Constellation Game, provides a situation to trigger helping behaviors by requiring them to be aware of other players’ difficulties. The feasibility of these activities has been confirmed in this study through viability tests with the school’s children with ASD and/or ID.

Depth detection in interactive projection system based on one-shot black-and-white stripe pattern.

A novel method enabling estimation of not only the screen surface as the conventional one, but the depth information from two-dimensional coordinates in an interactive projection system was proposed in this research. In this method, a one-shot black-and-white stripe pattern from a projector is projected on a screen plane, where the deformed pattern is captured by a charge-coupled device camera. An algorithm based on object/shadow simultaneous detection is proposed for fulfillment of the correspondence. The depth information of the object is then calculated using the triangulation principle. This technology provides a more direct feeling of virtual interaction in three dimensions without using auxiliary equipment or a special screen as interaction proxies. Simulation and experiments are carried out and the results verified the effectiveness of this method in depth detection.

ウエアラブルフロアインタラクションシステムにおける足モデル導入による足検出の向上の研究

Our previous study designed interactive floor projection system with hands and toes input in terms of quick and light input system. This system lets user interact with floor projection by pointing the finger or stepping on projection image. It allows to realize floor interaction without using even a single hands.

Linear Interpolation Transition of Character Animation for Immediate 3D Response to User Motion

The purpose of this research is to study methods for performing transition that have visual representation of corresponding animations with no bounce in subsequently recognized user information when attempting to interact with a virtual 3D character in real-time using user motion. If the transitions of the animation are needed owing to a variety of external environments, continuous recognition of user information is required to correspond to the motion. The proposed method includes linear interpolation of the transition using cross-fades and blending techniques. The normalized playing time of the source animation was utilized for automatically calculating the transition interpolation length of the target animation and also as the criteria in selecting the crossfades and blending techniques. In particular, in the case of blending, the weighting value based on the degree of similarity between two animations is used as a blending parameter. Accordingly, transitions for visually excellent animation are performed on interactive holographic projection systems.

A method for image-based shadow interaction with virtual objects

Abstract A lot of researchers have been investigating interactive portable projection systems such as a mini-projector. In addition, in exhibition halls and museums, there is a trend toward using interactive projection systems to make viewing more exciting and impressive. They can also be applied in the field of art, for example, in creating shadow plays. The key idea of the interactive portable projection systems is to recognize the user's gesture in real-time. In this paper, a vision-based shadow gesture recognition method is proposed for interactive projection systems. The gesture recognition method is based on the screen image obtained by a single web camera. The method separates only the shadow area by combining the binary image with an input image using a learning algorithm that isolates the background from the input image. The region of interest is recognized with labeling the shadow of separated regions, and then hand shadows are isolated using the defect, convex hull, and moment of each region. To distinguish hand gestures, Hu's invariant moment method is used. An optical flow algorithm is used for tracking the fingertip. Using this method, a few interactive applications are developed, which are presented in this paper.

Bare-fingers Touch Detection by the Button's Distortion in a Projector–Camera System

In this paper, we propose a novel interactive projection system (IPS), which enables bare-finger touch interaction on regular planar surfaces (e.g., walls, tables), with only one standard camera and one projector. The challenge of bare-finger touch detection is recovering the touching information just from the 2-D image captured by the camera. In our method, the graphical user interface (GUI) button is projected on the surface and is distorted by the finger when clicking it, and there is a significant positive correlation between the button's distortion and the finger's height to the surface. Therefore, we propose a novel, fast, and robust algorithm, which takes advantage of the button's distortion to detect the touch action. The proposed touch detection algorithm is performed in three stages: 1) region of interest extraction through a homography mapping, by which the computational complexity of the following processing is reduced; 2) the button's distortion detection using a special edge detection algorithm, which greatly reduces the errors due to the influence of the finger's shadows and edges; and 3) touch action judgment by the button's distortion. Several applications (e.g., virtual keyboard, power point viewing), which use the proposed touch detection method based on the buttons, are shown in this paper. An evaluation is performed on the virtual keyboard and the results demonstrate that the proposed approach can detect bare-finger touch in real time with the missed detection rate of 1.00%, false detection rate of 2.08%, and touch detection rate of 96.92% at the typical projected distance.

Using homography relationship for auto-calibration in mobile smart-project device system

As technology advances, mobile devices have become indispensable 3C products. Smart phones and tablet computers have become the favorite products of technology, and they are commonly seen everywhere. In addition to making phone calls and sending messages, people also use smart phones and tablet computers to browse the Internet, listen to music, take videos and pictures, and play games, all of which have become an integral part of people's lives. In addition, according to a market survey, the most expected function of future smart phones by consumers is the projection. However, the projection will accelerate the battery power consumption. In this paper, we conducted an auto-calibration based on homography relationship in mobile smart-project device system to take advantage of the capability of the projector to project an image onto any opaque and unflat plane, thus breaking through the restricted display interfaces of mobile devices. Moreover, we also proposed the method that named FDPA (standing for Fast Detecting Projection Area) to reduce the battery power consumption via decreasing the calculation cost of the auto-calibration. The experimental results show that FDPA is able to improve the dynamic projection performance and to reduce battery power consumption in the mobile smart-projector device system. This was expected to help further develop an interactive projection system between mobile devices, adding diversity to mobile device applications and development.

Display systems and registration methods for augmented reality applications

Introductions are given to various display systems and corresponding registration methods developed and realized at Beijing Institute of Technology for augmented reality applications. An interactive projection system is used in a theatrical performance, and the registration between the virtual scene (projected image) and the real objects (dancers) is realized through a video camera working in the near infrared waveband. An ultra-light wide-angle head-mounted display system is designed for an interactive exhibition item in a scientific museum, and a multi-user six-degree-of-freedom tracking system is developed for the indoor registration, which uses infrared markers projected on the ceiling and a camera fixed on top of each head-mounted display. A fixed-position viewing system is developed for the on-site digital reconstruction of a historical site, which provides visitors with the visualization of the original magnificent buildings on the natural field of the ruins. The azimuth and pitch of the viewing system are detected by two rotary encoders. A volumetric 3D display based on multi-angle projection is applied in a navigation system for endoscopic sinus surgery. A 3D digital scanner is employed to scan the skin on the patient's head for the alignment between the patient and the 3D model from his CT or MRI images. The registration of the surgical tool is also achieved by optical means through tracking cameras.

A Pen-Based Interactive Projection System Based on the Infrared Scanning Projector

Nowadays, how to make a human-computer interaction system which can take into account large-area display, high precision tracking and multi-user operation at the same time is a hot research topic. This paper presents a novel low cost pen-based projective interaction system that can write or draw like writing brush. The system is composed of infrared scanning projector, digital pen, projector, screen, wireless receiver module and computer. The infrared scanning projector is designed to achieve fast projection, which is to track the pen's two dimensional location accurately along with the digital pen. The system has the advantage of multi-user supported, high tracking precision, and has the function of identity recognition and writing calligraphy.

IpProjector: Designs and Techniques for Geometry-Based Interactive Applications Using a Portable Projector

We propose an interactive projection system for a virtual studio setup using a single self-contained and portable projection device. The system is named ipProjector, which stands for Interactive Portable Projector. Projection allows special effects of a virtual studio to be seen by live audiences in real time. The portable device supports 360-degree shooting and projecting angles and is easy to be integrated with an existing studio setup. We focus on two fundamental requirements of the system and their implementations. First, nonintrusive projection is performed to ensure that the special effect projections and the environment analysis (for locating the target actors or objects) can be performed simultaneously in real time. Our approach uses Digital Light Processing technology, color wheel analysis, and nearest-neighbor search algorithm. Second, a paired projector-camera system is geometrically calibrated with two alternative setups. The first uses a motion sensor for real-time geometric calibration, and the second uses a beam splitter for scene-independent geometric calibration. Based on a small-scale laboratory setting, experiments were conducted to evaluate the geometric accuracy of the proposed approaches, and an application was built to demonstrate the proposed ipProjector concept. Techniques of special effect rendering are not concerned in this paper.

Computer Controlled Interactive Motion and Still Image Film Projection System for Vocational Education for the Deaf

A severe problem exists for the public non-residential school systems in delivery of vocational instruction of the deaf, and management for vocational education of the deaf. The problem arises from the unique requirement of the deaf to have instruction delivered visually; lack of sufficient resource personnel in vocational education who are skilled in manual communication; and lack of vocational materials of instruction which were specifically developed to accommodate to the condition of deafness. This paper describes a micro-processor controlled instruction delivery system now being developed to better provide delivery of vocational materials of instruction to the deaf, which incorporates film cassette, frame or sequence addressable, color motion or still visuals under computer program control for local or remote delivery. It also describes an authoring capability for development and validation of vocational materials of instruction, together with an algorithmic basis for instruction module design. Finally, it describes a program which could result in early and extensive use of these materials by schools for the deaf and special education departments serving non-residential deaf and hearing impaired students.