Automatic Camera Control Research Articles

Rapid and Automatic Image Acquisition System for Structural Surface Defects of High-Speed Rail Tunnels

Continuous high-speed (>80 km/h) tunnel-image detection introduces new challenges to defect image acquisition, including machine recognition of entry and exit from a tunnel, storage of massive image data streams, image distortion due to surface photography, and encoder errors. In this study, an automatic image acquisition system is designed, and several critical technologies are proposed for high-speed rail tunnels. The system realizes quick tunnel identification and automatic camera control and uses the proposed line-scan software-matching method to obtain accurate images. Before storage, the real-time modification of the distorted images is implemented using an error-correction algorithm. An alternative mapping storage algorithm is proposed to improve the efficiency and stability of long-term storage. The test results show that the proposed method effectively reduces photographic errors. The lateral pixel-error rate of the corrected image is 1.60%, which is 10 times lower than that of the pre-correction, and the error rate of the longitudinal image is controlled within 10% when the system is moving at a variable speed and within 1% when it is moving at a constant speed. Furthermore, experiments have proven that the alternate mapping storage algorithm improves the storage efficiency of RAID by 50% and ensures data integrity; storage is accomplished by 8 HDDs at a camera throughput of 1.52 GB/s. This study will contribute to improvements in the speed and accuracy of tunnel defect-image detection.

Representational Requirements for a Plan Based Approach to Automated Camera Control

Automated camera control has been an active area of research for a number of years. The problem has been addressed in the Graphics, AI and Game communities from different perspectives. The main focus of the research in the Graphics community has been frame composition and coherence. The AI community has focused on intelligent shot selection, and the Games community strives for real- time cinematic camera control. While the proposed solutions in each of these fields are promising, there has not been much effort spent on listing out the requirements of an intelligent camera control system and how these can be satisfied through a combination of approaches taken from these different fields. This paper attempts to list out the representational requirements with a view of finding a unifying representation for combining these disparate approaches. We show how a plan based approach can capture some of these requirements and it can be connected to a geometric constraint solver for camera placement.

Remote and Automated Camera Control Technology

The basic approaches to remote control of the movie camera, including motion control, were explored. The technical advantages of using such technologies in advertising and feature film production are analyzed. The perspective directions of development for such technologies in the conditions of Ukrainian film production are shown.

Real‐time Anticipation of Occlusions for Automated Camera Control in Toric Space

AbstractEfficient visibility computation is a prominent requirement when designing automated camera control techniques for dynamic 3D environments; computer games, interactive storytelling or 3D media applications all need to track 3D entities while ensuring their visibility and delivering a smooth cinematic experience. Addressing this problem requires to sample a large set of potential camera positions and estimate visibility for each of them, which in practice is intractable despite the efficiency of ray‐casting techniques on recent platforms. In this work, we introduce a novel GPU‐rendering technique to efficiently compute occlusions of tracked targets in Toric Space coordinates – a parametric space designed for cinematic camera control. We then rely on this occlusion evaluation to derive an anticipation map predicting occlusions for a continuous set of cameras over a user‐defined time window. We finally design a camera motion strategy exploiting this anticipation map to minimize the occlusions of tracked entities over time. The key features of our approach are demonstrated through comparison with traditionally used ray‐casting on benchmark scenes, and through an integration in multiple game‐like 3D scenes with heavy, sparse and dense occluders.

LAMOST CCD camera-control system based on RTS2

The Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST) is the largest existing spectroscopic survey telescope, having 32 scientific charge-coupled-device (CCD) cameras for acquiring spectra. Stability and automation of the camera-control software are essential, but cannot be provided by the existing system. The Remote Telescope System 2nd Version (RTS2) is an open-source and automatic observatory-control system. However, all previous RTS2 applications were developed for small telescopes. This paper focuses on implementation of an RTS2-based camera-control system for the 32 CCDs of LAMOST. A virtual camera module inherited from the RTS2 camera module is built as a device component working on the RTS2 framework. To improve the controllability and robustness, a virtualized layer is designed using the master-slave software paradigm, and the virtual camera module is mapped to the 32 real cameras of LAMOST. The new system is deployed in the actual environment and experimentally tested. Finally, multiple observations are conducted using this new RTS2-framework-based control system. The new camera-control system is found to satisfy the requirements for automatic camera control in LAMOST. This is the first time that RTS2 has been applied to a large telescope, and provides a referential solution for full RTS2 introduction to the LAMOST observatory control system.

Focusness Extraction from Images for Automatic Camera Control by Measuring Objectness

Current digital cameras have various automatic control systems. In automatic camera systems, extracting focusness from an image is a very important problem. Automatic extraction of the main subject makes taking photos very easy, even for an amateur photographer. Methods have been proposed to evaluate focusness by visual saliency, which assume that an area with high saliency also has high focusness. However, various differences exist between focusness and saliency. In this study, the authors compare the values between focusness and saliency maps. They evaluate the focusness of 80 images in an image evaluation experiment with 20 observers. Saliency maps are calculated using six conventional algorithms. They show that the individual variations of focusness are insignificant in images that include only one major object. Furthermore, the authors apply a GIST feature to the saliency method by using a center‐surround histogram and extract focusness from images with high accuracy.

Virtual Camera Control Based on Cinematographic Rules of Camera Shot and Camera Angle

This research is focused on developing automatic virtual camera control in 3D video game to support a filmmaking based on machinima. Rule-based virtual camera control defines the camera position and the angle. A virtual camera is developed by representing camera shot and camera angle since it can control its position and rotation automatically. Knowledge acquisition for camera shot and camera angle is conducted by simulating virtual camera to the character based on rules of camera shot which are close-up, medium-shot, and long-shot, and rules of camera angle which are eye-level, low angle and high angle. The constrained areas are represented by defining paths. The paths have the function of track references to define position and rotation for the virtual camera, so the virtual camera can capture character vertically, horizontally, and diagonally. Data of constrained areas are representation of camera shot and camera angle rules. The data are used as knowledge base and rules in the term of perspective packages. The virtual camera is generated using Genetic algorithm by randomizing a number of position and rotation value as initial population, then the system calculates the value to find ideal virtual camera which can fulfil the rules in perspective packages. An implementation of the research is conducted by developing a game in which the automatic virtual camera is generated to produce a short animation with duration of 120 seconds. The rules of transition are implemented using different cut as type of transition in every 4 seconds, which means that there are 30 cut to produce. The result shows that the method used in this research reflects the expectation. All the 30 cuts produced in the game are captured using automatic virtual camera. With frame rate at 40-60 fps, the virtual camera is generated in 2 to 16 generations, with duration from 16 to 37 milliseconds.

Intuitive and efficient camera control with the toric space

A large range of computer graphics applications such as data visualization or virtual movie production require users to position and move viewpoints in 3D scenes to effectively convey visual information or tell stories. The desired viewpoints and camera paths are required to satisfy a number of visual properties ( e.g. size, vantage angle, visibility, and on-screen position of targets). Yet, existing camera manipulation tools only provide limited interaction methods and automated techniques remain computationally expensive. In this work, we introduce the Toric space , a novel and compact representation for intuitive and efficient virtual camera control. We first show how visual properties are expressed in this Toric space and propose an efficient interval-based search technique for automated viewpoint computation. We then derive a novel screen-space manipulation technique that provides intuitive and real-time control of visual properties. Finally, we propose an effective viewpoint interpolation technique which ensures the continuity of visual properties along the generated paths. The proposed approach (i) performs better than existing automated viewpoint computation techniques in terms of speed and precision, (ii) provides a screen-space manipulation tool that is more efficient than classical manipulators and easier to use for beginners, and (iii) enables the creation of complex camera motions such as long takes in a very short time and in a controllable way. As a result, the approach should quickly find its place in a number of applications that require interactive or automated camera control such as 3D modelers, navigation tools or 3D games.

Automatic camera control meets emergency simulations: An application to aviation safety

Computer-based simulations of emergencies increasingly adopt 3D graphics to visualize results and thus generate complex dynamic 3D scenes with many potentially parallel events that affect large groups of virtual characters. To understand the portrayed scenario, a viewer could interactively control a flying camera or switch among a set of virtual cameras that have been previously placed at modeling time. The first solution imposes a cognitive load on the viewer that can distract him/her from the analysis task, and (s)he might miss events while moving the camera. The second solution requires additional work in the modeling phase, and even a very large number of cameras could fail to correctly frame events because of dynamic occlusions. More sophisticated automatic camera control methods could help, but the methods in the literature are designed for sequential dialogue-like events that involve at most two or three characters and therefore would not work. In this paper, we present a fully automated, real-time system that is able to monitor events in emergency simulations, select relevant events based on user-provided filtering rules, and control a virtual camera such that the events of interest are properly presented to the viewer. To illustrate how the system works in practice, we also describe the first application of automatic camera control to the domain of aviation safety.

Camera Gesture Estimation for Confirming View Point of Virtual 3D Face Model

Camera gesture information is important for making virtual 3D face model synchronize to the real face at view point, in other areas such as: human-computer interface, 3D object estimation, automatic camera control etc. Camera gesture is also very important. In this paper, we will introduce an algorithm to detect camera gesture from a single face image, through our algorithm, camera rotation angles in both x-axis and y-axis and distance can be gained to confirm camera gesture. Then we can set the camera gesture information to view point of virtual 3D face model. This can make can make virtual 3D face model synchronize to the real face at view point. Experiment results demonstrate the validity of our algorithm and the correct estimation rate is accredited for applying it into practice.

A depth camera based approach for automatic control of video cameras in lecture halls

Purpose – The purpose of this study is to present a depth information-based solution for automatic camera control, depending on the presenter’s moving positions. Talks, presentations and lectures are often captured on video to give a broad audience the possibility to (re-)access the content. As presenters are often moving around during a talk, it is necessary to steer recording cameras. Design/methodology/approach – We use depth information from Kinect to implement a prototypical application to automatically steer multiple cameras for recording a talk. Findings – We present our experiences with the system during actual lectures at a university. We found out that Kinect is applicable for tracking a presenter during a talk robustly. Nevertheless, our prototypical solution reveals potential for improvements, which we discuss in our future work section. Originality/value – Tracking a presenter is based on a skeleton model extracted from depth information instead of using two-dimensional (2D) motion- or brightness-based image processing techniques. The solution uses a scalable networking architecture based on publish/subscribe messaging for controlling multiple video cameras.

LectureSight: an open source system for automatic camera control for lecture recordings

Purpose – The purpose of this work is to present a prototype of the system and the results from a technical evaluation and a study on possible effects of recordings with active camera control on the learner. An increasing number of higher education institutions have adopted the lecture recording technology in the past decade. Even though some solutions already show a very high degree of automation, active camera control can still only be realized with the use of human labor. Aiming to fill this gap, the LectureSight project is developing a free solution for active autonomous camera control for presentation recordings. The system uses a monocular overview camera to analyze the scene. Adopters can formulate camera control strategies in a simple scripting language to adjust the system’s behavior to the specific characteristics of a presentation site. Design/methodology/approach – The system is based on a highly modularized architecture to make it easily extendible. The prototype has been tested in a seminar room and a large lecture hall. Furthermore, a study was conducted in which students from two universities prepared for a simulated exam with an ordinary lecture recording and a recording produced with the LectureSight technology. Findings – The technical evaluation showed a good performance of the prototype but also revealed some technical constraints. The results of the psychological study give evidence that the learner might benefit from lecture videos in which the camera follows the presenter so that gestures and facial expression are easily perceptible. Originality/value – The LectureSight project is the first open-source initiative to care about the topic of camera control for presentation recordings. This opens way for other projects building upon the LectureSight architecture. The simulated exam study gave evidence of a beneficial effect on students learning success and needs to be reproduced. Also, if the effect is proven to be consistent, the mechanism behind it is worth to be investigated further.

Online control of active camera networks for computer vision tasks

Large networks of cameras have been increasingly employed to capture dynamic events for tasks such as surveillance and training. When using active cameras to capture events distributed throughout a large area, human control becomes impractical and unreliable. This has led to the development of automated approaches for online camera control. We introduce a new automated camera control approach that consists of a stochastic performance metric and a constrained optimization method . The metric quantifies the uncertainty in the state of multiple points on each target. It uses state-space methods with stochastic models of target dynamics and camera measurements. It can account for occlusions, accommodate requirements specific to the algorithms used to process the images, and incorporate other factors that can affect their results. The optimization explores the space of camera configurations over time under constraints associated with the cameras, the predicted target trajectories, and the image processing algorithms. The approach can be applied to conventional surveillance tasks (e.g., tracking or face recognition), as well as tasks employing more complex computer vision methods (e.g., markerless motion capture or 3D reconstruction).

Video Surveillance Over Wireless Sensor and Actuator Networks Using Active Cameras

Although there has been much work focused on the camera control issue on keeping tracking a target of interest, few has been done on jointly considering the video coding, video transmission, and camera control for effective and efficient video surveillance over wireless sensor and actuator networks (WSAN). In this work, we propose a framework for real-time video surveillance with pan-tilt cameras where the video coding and transmission as well as the automated camera control are jointly optimized by taking into account the surveillance video quality requirement and the resource constraint of WSANs. The main contributions of this work are: i) an automated camera control method is developed for moving target tracking based on the received surveillance video clip in consideration of the impact of video transmission delay on camera control decision making; ii) a content-aware video coding and transmission scheme is investigated to save network node resource and maximize the received video quality under the delay constraint of moving target monitoring. Both theoretical and experimental results demonstrate the superior performance of the proposed optimization framework over existing systems.

Automatic camera control in virtual environments augmented using multiple sparse videos

Automated virtual camera control has been widely used in animation and interactive virtual environments. We have developed a multiple sparse camera based free view video system prototype that allows users to control the position and orientation of a virtual camera, enabling the observation of a real scene in three dimensions (3D) from any desired viewpoint. Automatic camera control can be activated to follow selected objects by the user. Our method combines a simple geometric model of the scene composed of planes (virtual environment), augmented with visual information from the cameras and pre-computed tracking information of moving targets to generate novel perspective corrected 3D views of the virtual camera and moving objects. To achieve real-time rendering performance, view-dependent textured mapped billboards are used to render the moving objects at their correct locations and foreground masks are used to remove the moving objects from the projected video streams. The current prototype runs on a PC with a common graphics card and can generate virtual 2D views from three cameras of resolution 768×576 with several moving objects at about 11fps.

Providing Dynamic Visual Information for Collaborative Tasks: Experiments With Automatic Camera Control

One possibility presented by novel communication technologies is the ability for remotely located experts to provide guidance to others who are performing difficult technical tasks in the real world, such as medical procedures or engine repair. In these scenarios, video views and other visual information seem likely to be useful in the ongoing negotiation of shared understanding, or common ground, but actual results with experimental systems have been mixed. One difficulty in designing these systems is achieving a balance between close-up shots that allow for discussion of detail and wide shots that allow for orientation or establishing a mutual point of focus in a larger space. Achieving this balance can be difficult without disorienting or overloading task participants. In this article we present results from two experiments involving three automated camera control systems for remote repair tasks. Results show that a system providing both detailed and overview information was superior to systems providing only one or the other in terms of performance but that some participants preferred the detail-only system.

The Virtual Director: a Correlation‐Based Online Viewing of Human Motion

AbstractAutomatic camera control for scenes depicting human motion is an imperative topic in motion capture base animation, computer games, and other animation based fields. This challenging control problem is complex and combines both geometric constraints, visibility requirements, and aesthetic elements. Therefore, existing optimization‐based approaches for human action overview are often too demanding for online computation.In this paper, we introduce an effective automatic camera control which is extremely efficient and allows online performance. Rather than optimizing a complex quality measurement, at each time it selects one active camera from a multitude of cameras that render the dynamic scene. The selection is based on the correlation between each view stream and the human motion in the scene. Two factors allow for rapid selection among tens of candidate views in real‐time, even for complex multi‐character scenes: the efficient rendering of the multitude of view streams, and optimized calculations of the correlations using modified CCA. In addition to the method's simplicity and speed, it exhibits good agreement with both cinematic idioms and previous human motion camera control work. Our evaluations show that the method is able to cope with the challenges put forth by severe occlusions, multiple characters and complex scenes.

An information theoretic approach to camera control for crowded scenes

Navigation and monitoring of large and crowded virtual environments is a challenging task and requires intuitive camera control techniques to assist users. In this paper, we present a novel automatic camera control technique providing a scene analysis framework based on information theory. The developed framework contains a probabilistic model of the scene to build entropy and expectancy maps. These maps are utilized to find interest points which represent either characteristic behaviors of the crowd or novel events occurring in the scene. After an interest point is chosen, the camera is updated accordingly to display this point. We tested our model in a crowd simulation environment and it performed successfully. Our method can be integrated into existent camera control modules in computer games, crowd simulations and movie pre-visualization applications.

Virtual world explorations by using topological and semantic knowledge

This paper is dedicated to virtual world exploration techniques. Automatic camera control is important in many fields as computational geometry, visual servoing, robot motion, graph drawing, etc. The paper introduces a high-level camera controlling approach in virtual environments. The proposed method is related to real-time 3D scene exploration and is made of two steps. In the first step, a set of good viewpoints is chosen to give the user a maximum knowledge of the scene. The second step uses the viewpoints to compute a camera path between them. Finally, we define a notion of semantic distance between objects of the scene to improve the approach.

Automated camerawork for capturing desktop presentations

A novel automated camera control method for capturing desktop presentations introduced. For this purpose, typical features of shots and their camerawork that appear frequently in TV programs are discussed. To realise those features in this automated video capturing system, the purpose of camerawork is classified from two points of view: target and aspect-of-target. Then, the correspondence between the classification and typical shots and cameraworks is considered. The virtual-frame control algorithm based on this idea is proposed, and the implementation in the video production system. Experimental results are shown that verified this method through two kinds of experiments: virtual video capturing using CG animations and real video capturing of real presentations.