Live Camera View Research Articles

Application of Industrial Remote Access Control Laboratory

An industrial-scale remote access control laboratory system was developed to address the potential lack of practical laboratory experience for Control and Automation Engineering (CAE) students at Istanbul Technical University (ITU) in response to the coronavirus pandemic 2019 (COVID-2019). The system consists of six fundamental control theory experiments and aims to test fundamental control engineering topics such as mathematical modeling, system identification, controller design, and design comparison. These experiments are integrated into a remote access system using industry 4.0 methodologies, allowing students to use the experiments remotely over web-based user interfaces. The system has necessary capabilities such as remote control, online experimentation, graphs and charts, live camera view, data storage, accessibility options, and security for robust interaction with students. Two surveys were conducted to collect feedback from students who have not used the system and those who have used it for at least a semester. The results of the first survey showed that students appreciate the opportunity of remote-access laboratory usage on control experiments and would consider using it. The results of the second survey demonstrated that the system fulfills the varied requirements of students and received high overall satisfaction. Overall, the demand for an industrial-scale remote access control laboratory has been met, driven by the COVID-19 pandemic and the dynamic environment of engineering education.

Upper-body haptic system for snake robot teleoperation in pipelines

Snake robots have shown a great potential for operations in confined workplaces that are less accessible or dangerous to human workers, such as the in-pipe inspection. However, the snake robot teleoperation remains a nontrivial task due to the unique locomotion mechanism (e.g., helical motion) and the constraints of the workplaces including the low visibility and indistinguishable features. Most snake robot feedback systems are based on the live camera view only. It is hard for the human operator to develop a correct spatial understanding of the remote workplace, leading to problems such as disorientation and motion sickness in snake robot teleoperation. This study designs and evaluates an innovative haptic assistant system for snake robot teleoperation in the in-pipe inspection. An upper-body haptic suit with 40 vibrators on both the front and back sides of the human operator was developed to generate haptic feedback corresponding to the bottom and up sides of the snake robot, transferring the egocentric sensation of the snake robot to the human operator. A human-subject experiment (n = 31) was performed to evaluate the efficacy of the developed system. The results indicate that the proposed haptic assistant system outperformed other feedback systems in terms of both task performance and subjective workload and motion sickness evaluations. It inspires new control and feedback designs for the future snake robot in industrial operations.

Mobile augmented reality based indoor map for improving geo-visualization.

Unlike traditional visualization methods, augmented reality (AR) inserts virtual objects and information directly into digital representations of the real world, which makes these objects and data more easily understood and interactive. The integration of AR and GIS is a promising way to display spatial information in context. However, most existing AR-GIS applications only provide local spatial information in a fixed location, which is exposed to a set of problems, limited legibility, information clutter and the incomplete spatial relationships. In addition, the indoor space structure is complex and GPS is unavailable, so that indoor AR systems are further impeded by the limited capacity of these systems to detect and display location and semantic information. To address this problem, the localization technique for tracking the camera positions was fused by Bluetooth low energy (BLE) and pedestrian dead reckoning (PDR). The multi-sensor fusion-based algorithm employs a particle filter. Based on the direction and position of the phone, the spatial information is automatically registered onto a live camera view. The proposed algorithm extracts and matches a bounding box of the indoor map to a real world scene. Finally, the indoor map and semantic information were rendered into the real world, based on the real-time computed spatial relationship between the indoor map and live camera view. Experimental results demonstrate that the average positioning error of our approach is 1.47 m, and 80% of proposed method error is within approximately 1.8 m. The positioning result can effectively support that AR and indoor map fusion technique links rich indoor spatial information to real world scenes. The method is not only suitable for traditional tasks related to indoor navigation, but it is also promising method for crowdsourcing data collection and indoor map reconstruction.

An Augmented Reality Multimedia (ARM): A Dynamic Multimedia Encyclopedia on Mobile Device

In the recent days, there are rapid advancements in the web 2.0 contents (texts, images and videos) and in the computational complexity. The next generation in computing is predictably in the mobile arena. A lot of applications are being developed to cater the needs of mobile device users. This research is based on the question: How can we benefit from the rapidly proliferating web 2.0 contents in the mobile device with staggering technological developments? This paper presents a surprisingly impressive novel dynamic multimedia encyclopedia, multimedia, that augments reality on the mobile device. This system supersedes the traditional texts or maps retrieved. A group of estimation and sensors to track objects and keep information on them used. An augmented reality based multimedia encyclopedia on mobile device which is dynamic is presented. It is dynamic and mines resources on the web. We demonstrate how this device can be utilized to bring relevant information to users in a much more enriched manner than traditional texts. ARM is dynamic and is distinctive in 3 ways. It fetches multimedia contents for users through automated intuition and dynamically updates content from the web. It uses recent multimedia content to produce answers which are versatile. New mediums like videos & languages can be incorporated into the framework. This system takes advantage of the mobile camera, location and orientation sensors to augment reality. Live camera view and retrieve dynamic data. A Combination of algorithms is used to analyze and demonstrate our ARM.

Hierarchical 2.5-D Scene Alignment for Change Detection With Large Viewpoint Differences

Change detection from mobile platforms is a relevant topic in the field of intelligent vehicles and has many applications, such as countering improvised explosive devices (C-IED). Existing real-time C-IED systems are not robust against large viewpoint differences, which are unavoidable under realistic operating conditions in outdoor environments. To address this, we proposes a new hierarchical 2.5-D scene-alignment algorithm. First, the 3-D ground surface of the historic scene is reconstructed by polygons, onto which historic image-based texture is projected. By estimating the 3-D transformation between historic and live camera views, the historic scene can be rendered as if seen from the live camera viewpoint. To compensate for 3-D alignment and reconstruction imperfections, local pixel-accurate registration refinement is performed in 2-D. The proposed real-time 2.5-D method thereby combines the accuracy of a 2-D local image registration with the robustness of 3-D scene alignment. It was found that the resulting change detection system detects small changes of only $18 \times 18 \times 9 \text{ cm}$ at distances of 60 meters under large trajectory deviations of up to 2.5 meters.

An Efficient Method of Image-Sound Conversion Based on IFFT for Vision Aid for the Blind

The vOICe technology is a kind of vision aid technology for the blind, which offers the experience of live camera views through image-sound conversion. To reduce the computational complexity of image-sound conversion, a novel image-sound conversion method is proposed in the paper. The new method takes each column of an image as the Discrete Fourier Transform (DFT) of an audio signal, and inverse Fast Fourier Transform (IFFT) is utilized to implement the inverse DFT and calculate the converted audio signal. The final result represent that this method is more effectively and has a better real time performance. Vision is the most important way for human to perceive the outside world, more than 80% of the information we obtain is derived from vision. The visually impaired has difficulty to gain information from eyes because of the impairment, meanwhile it also gives living or economic

An Augmented Reality System for Epidural Anesthesia (AREA): Prepuncture Identification of Vertebrae

We propose an augmented reality system to identify lumbar vertebral levels to assist in spinal needle insertion for epidural anesthesia. These procedures require careful placement of a needle to ensure effective delivery of anesthetics and to avoid damaging sensitive tissue such as nerves. In this system, a trinocular camera tracks an ultrasound transducer during the acquisition of a sequence of B-mode images. The system generates an ultrasound panorama image of the lumbar spine, automatically identifies the lumbar levels in the panorama image, and overlays the identified levels on a live camera view of the patient's back. Validation is performed to test the accuracy of panorama generation, lumbar level identification, overall system accuracy, and the effect of changes in the curvature of the spine during the examination. The results from 17 subjects demonstrate the feasibility and capability of achieving an error within clinically acceptable range for epidural anaesthesia.

How to See the Invisible

The article discusses a mobile device application (app), called augmented reality (AR), which combines live-camera views with superimposed informational graphics. The author notes that AR apps convert the cell phone into a kind of magic looking glass that identifies physical objects in the world of its users. Examples of AR apps that aid individuals when they are traveling or commuting are presented, including the New York Nearest Subway app and the Metro AR, which allows users to look at the ground and see colorful arrows showing which subway lines are underneath their feet, the Augmented Car Finder app to help individuals locate their automobile, and the SpotCrime app that shows what crimes have recently been committed in the neighborhoods around the user.

Indirect augmented reality

Developing augmented reality (AR) applications for mobile devices and outdoor environments has historically required a number of technical trade-offs related to tracking. One approach is to rely on computer vision which provides very accurate tracking, but can be brittle, and limits the generality of the application. Another approach is to rely on sensor-based tracking which enables widespread use, but at the cost of generally poor tracking performance. In this paper we present and evaluate a new approach, which we call Indirect AR, that enables perfect alignment of virtual content in a much greater number of application scenarios.To achieve this improved performance we replace the live camera view used in video see through AR with a previously captured panoramic image. By doing this we improve the perceived quality of the tracking while still maintaining a similar overall experience. There are some limitations of this technique, however, related to the use of panoramas. We evaluate these boundaries conditions on both a performance and experiential basis through two user studies. The result of these studies indicates that users preferred Indirect AR over traditional AR in most conditions, and when conditions do degrade to the point the experience changes, Indirect AR can still be a very useful tool in many outdoor application scenarios.

The PumaPaint Project

The PumaPaint Project is an online robot that allows World Wide Web users to create original artwork. This paper describes the PumaPaint Project at two locations: the original site at Wilkes University and the site at Roger Williams University. Each site allows control of a PUMA robot equipped with four paintbrushes, jars of red, green, blue and yellow paint and white paper attached to a vertical easel. A Java™ interface executing within a web browser allows interactive control of the robot. This interface contains two windows showing live camera views of the easel and various controls for connecting to the robot, viewing the task status and controlling the painting task. The original site operated from June 1998 to March 2000 with approximately 25,000 unique-addressed machines downloading the interface to produce about 500 canvases. The new site has been in operation since August 2002. This paper discusses the author's experiences in operating the original site, and the motivation for and the challenges of reviving the site in its current location.