Active Camera Research Articles

多スタート局所探索法を用いた多数人物追跡のための複数能動カメラの同時視線プランニング

多スタート局所探索法を用いた多数人物追跡のための複数能動カメラの同時視線プランニング

2A2-F07 複数の平行線パターンを用いた視覚移動ロボットの自己位置推定(移動ロボットのための視覚)

We propose a new self-localization method for a mobile robot equipped with a fixed-viewpoint active camera by observing multiple parallel lines patterns with the camera. The method utilizes the technique for estimating location and posture of the camera from one parallel lines pattern which was previously developed by our research group. The method switches from a certain parallel lines pattern used for self-localization to another parallel lines pattern while the robot is moving in the environment. After the new parallel lines pattern is selected, the method keeps estimating location and posture of the camera from the parallel lines pattern. Our method can estimate location and posture of the robot and the parallel lines patterns in the world coordinate system fixed to the initial parallel lines pattern by repeating the switch of parallel lines patterns and the self-localization based on the parallel line pattern. Through the experiment of the self-localization of the real mobile robot, we confirmed the validity of the proposed method.

Surveillance camera scheduling: a virtual vision approach

We present a surveillance system, comprising wide field-of-view (FOV) passive cameras and pan/tilt/zoom (PTZ) active cameras, which automatically captures high-resolution videos of pedestrians as they move through a designated area. A wide-FOV static camera can track multiple pedestrians, while any PTZ active camera can capture high-quality videos of one pedestrian at a time. We formulate the multi-camera control strategy as an online scheduling problem and propose a solution that combines the information gathered by the wide-FOV cameras with weighted round-robin scheduling to guide the available PTZ cameras, such that each pedestrian is observed by at least one PTZ camera while in the designated area. A centerpiece of our work is the development and testing of experimental surveillance systems within a visually and behaviorally realistic virtual environment simulator. The simulator is valuable as our research would be more or less infeasible in the real world given the impediments to deploying and experimenting with appropriately complex camera sensor networks in large public spaces. In particular, we demonstrate our surveillance system in a virtual train station environment populated by autonomous, lifelike virtual pedestrians, wherein easily reconfigurable virtual cameras generate synthetic video feeds. The video streams emulate those generated by real surveillance cameras monitoring richly populated public spaces.

An augmented reality human–computer interface for object localization in a cognitive vision system

The European Cognitive Vision project VAMPIRE uses mobile AR-kits to interact with a visual active memory for teaching and retrieval purposes. This paper describes concept and technical realization of the used mobile AR-kits and discusses interactive learning and retrieval in office environments, and the active memory infrastructure. The focus is on 3D interaction for pointing in a scene coordinate system. This is achieved by 3D augmented pointing, which combines inside-out tracking for head pose recovery and 3D stereo human–computer interaction. Experimental evaluation shows that the accuracy of this 3D cursor is within a few centimeters, which is sufficient to point at an object in an office. Finally, an application of the cursor in VAMPIRE is presented, where in addition to the mobile system, at least one stationary active camera is used to obtain different views of an object. There are many potential applications, for example an improved view-based object recognition.

Human tracking using the temporally averaged silhouette with an active camera

AbstractThis research paper proposes an image processing algorithm for tracking nonrigid moving objects such as humans in outdoor locations. Since the camera itself moves, it is believed that an effective method extracts and tracks the movement object by using the differences in the flow against the background, rather than by using intensity‐based background subtraction. However, since nonrigid objects are tracked outdoors, the flow is partially lost due to variation in the shape and aspect, occlusion due to obstacles, passage through high‐contrast backgrounds, and other factors, so the tracking center of gravity is not stable, and tracking is often lost. In view of the above, tracking is carried out in the proposed method by using areas referred to as “temporally averaged silhouettes,” which are obtained by accumulating small areas extracted by the difference between the background and the flow while positioning the small areas in pixel units across a number of frames. Since the flow is observed over a long period of time instead of carrying out detailed spatial calculations, the area of the moving object can be stably calculated even if the flow partially disappears, so computational efficiency is good. The results of experiments using scenes with humans in motion confirm that stable tracking results that are equivalent to or better than cases in which the flow is calculated for the entire image (about 1200 points) are obtained by using a flow with a small number of points (average: 64 points) in the combined moving‐object and background areas. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(12): 69–81, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20490

Two-stage real-time head tracking by an active camera based on multimodal information

We propose a two-stage real-time tracking algorithm for an active camera system having pan-tilt-zoom functions. The algorithm is based on the assumption that a human head has an elliptical shape and that its model color histogram has been acquired in advance. The algorithm consists of two stages, a color-based convergence stage for fast and reliable tracking and a refinement stage for accurate tracking based on multimodal information. In the first color convergence stage, we roughly estimate the target position by using the mean-shift method based on the histogram similarity between the model and a candidate ellipse. To better predict the initial position for the mean shift, the global motion is compensated; to enhance reliability of the mean shift, the model histogram is appropriately updated by referring to the target histogram in the previous frame. In the subsequent refinement stage, we refine the position and size of the ellipse obtained at the first stage by using multimodal information such as color, shape, and quasi-spatial information. In particular, to quantify the quasi-spatial information, we use a spatial color histogram obtained by properly dividing the ellipse into two regions. Extensive experiments verify that the proposed algorithm robustly tracks the head, even when the subject moves quickly, the head size changes drastically, or the background has many clusters and/or distracting colors. Also, the proposed algorithm can perform real-time tracking with a processing speed of about 10 fps on a standard PC.

ArmillEye: Flexible Platform for Underwater Stereo Vision

The paper describes ArmillEye, a 3-degree of freedom (DOF) flexible hybrid platform designed for agile underwater stereoptic vision. Effective telecontrol systems of remote operated vehicles require active and dexterous camera support in order to allow the operator to easily and promptly change the point of view, also improving the virtual reconstruction of the environment in difficult operative conditions (dirtiness, turbulence, and partial occlusion). The same concepts hold for visual servoing of autonomous underwater vehicles. ArmillEye was designed for this specific application; it is based on the concept of using a parallel-hybrid mechanism architecture that, in principle, allows us to minimize the ad hoc waterproof boxes (generally only for cameras) while the actuators, fixed to the base of the mechanism, can be placed into the main body of the underwater vehicle. This concept was revealed effective and was previously proposed for underwater arms. The synthesis of ArmillEye followed the specific aims of visual telecontrol and servoing, specifying vision workspace, dexterity, and dynamics parameters. Two versions of ArmillEye are proposed: the first one with two cameras to obtain a steroptic vision by using two viewpoints (two rotational freedoms with a fixed tilt or pan axis and vergence); the second one with one camera operated to obtain a stereoptic vision by using one viewpoint (two rotational freedoms with a fixed tilt or pan axis and extrusion).

Pan–tilt–zoom camera calibration and high-resolution mosaic generation

In this paper, we discuss the problem of estimating parameters of a calibration model for active pan–tilt–zoom cameras. The variation of the intrinsic parameters of each camera over its full range of zoom settings is estimated through a two step procedure. We first determine the intrinsic parameters at the camera’s lowest zoom setting very accurately by capturing an extended panorama. The camera intrinsics and radial distortion parameters are then determined at discrete steps in a monotonically increasing zoom sequence that spans the full zoom range of the camera. Our model incorporates the variation of radial distortion with camera zoom. Both calibration phases are fully automatic and do not assume any knowledge of the scene structure. High-resolution calibrated panoramic mosaics are also computed during this process. These fully calibrated panoramas are represented as multi-resolution pyramids of cube-maps. We describe a hierarchical approach for building multiple levels of detail in panoramas, by aligning hundreds of images captured within a 1–12× zoom range. Results are shown from datasets captured from two types of pan–tilt–zoom cameras placed in an uncontrolled outdoor environment. The estimated camera intrinsics model along with the cube-maps provides a calibration reference for images captured on the fly by the active pan–tilt–zoom camera under operation making our approach promising for active camera network calibration.

Dynamic wide‐area scene visualization using an active camera

AbstractThis paper will discuss a method for presenting a surveillance image captured by an active camera such as one that actively tracks a moving subject as an image that is easily understood by an observer. Active camera surveillance images generally have a narrow field of view, and changes in the line‐of‐sight direction of an automatic camera make it difficult for an observer to intuitively understand the scene. However, enhancement of the observer's ability to comprehend the scene can be anticipated by adding a wide‐angle panoramic image to the periphery of the narrow‐field observed image and presenting it as an image that has a wider field of view. In order for an observer to understand conditions in a remote location, the observed image and the panoramic image must be integrated with geometric/photometric consistency. The present paper will discuss a method of geometric integration using a fixed‐viewpoint camera, and a method of photometric integration by modeling of brightness variation by eigenimage analysis. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(8): 101–112, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10160

Towards on-line saccade planning for high-resolution image sensing

This paper considers the problem of designing an active observer to plan a sequence of decisions regarding what target to look at, through a foveal-sensing action. We propose a framework in which a pan/tilt/zoom (PTZ) camera schedules saccades in order to acquire high resolution images (at least one) of as many moving targets as possible before they leave the scene. An intelligent choice of the order of sensing the targets can significantly reduce the total dead-time wasted by the active camera and, consequently, its cycle time. The grabbed images provide meaningful identification imagery of distant targets which are not recognizable in a wide angle view. We cast the whole problem as a particular kind of dynamic discrete optimization. In particular, we will show that the problem can be solved by modelling the attentional gaze control as a novel on-line dynamic vehicle routing problem (DVRP) with deadlines. Moreover we also show how multi-view geometry can be used for evaluating the cost of high resolution image sensing with a PTZ camera. Congestion analysis experiments are reported proving the effectiveness of the solution in acquiring high resolution images of a large number of moving targets in a wide area. The evaluation was conducted with a simulation using a dual camera system in a master-slave configuration. Camera performances are also empirically tested in order to validate how the manufacturer’s specification deviates from our model using an off-the-shelf PTZ camera.

Automated Alignment of Robotic Pan-Tilt Camera Units Using Vision

In this paper we show how to carry out an automatic alignment of a pan-tilt camera platform with its natural coordinate frame, using only images obtained from the cameras during controlled motion of the unit. An active camera in aligned orientation represents the zero position for each axis, and allows axis odometry to be referred to a fixed reference frame; such referral is otherwise only possible using mechanical means, such as end-stops, which cannot take account of the unknown relationship between the camera coordinate frame and its mounting. The algorithms presented involve the calculation of two-view transformations (homographies or epipolar geometry) between pairs of images related by controlled rotation about individual head axes. From these relationships, which can be calculated linearly or optimised iteratively, an invariant line to the motion can be extracted which represents an aligned viewing direction. We present methods for general and degenerate motion (translating or non-translating), and general and degenerate scenes (non-planar and planar, but otherwise unknown), which do not require knowledge of the camera calibration, and are resistant to lens distortion non-linearity.Detailed experimentation in simulation, and in real scenes, demonstrate the speed, accuracy, and robustness of the methods, with the advantages of applicability to a wide range circumstances and no need to involve calibration objects or complex motions. Accuracy of within half a degree can be achieved with a single motion, and we also show how to improve on this by incorporating images from further motions, using a natural extension of the basic algorithm.

移動人物を対象とする多方向顔画像取得と認識

Multi-directional face image acquisition of moving people and techniques for recognizing them are described. Our system uses an extra active camera in addition to a conventional fixed camera. When the fixed camera detects a person who moves toward the active camera, it zooms in towards the person's head and captures close-up multi-directional face images by using multi-directional-averaged-face templates. It captures 3.9 times as many multi-directional face images as the technique using the front averaged-face template only. Our technique substantially increases the chance of identifying a person in a looking-at-people system. Further more, the obtained face image is recognized by using the template matching technique. The recognition system showed a correct recognition rate of 86.0 % for the face images automatically obtained from people in experimental environments. Our technique would be useful for this system to recover people's names if face-recognition fails at a specific place such as a room entrance, a person steps out of view and quickly comes back into view, or an original ID is reinstated after occlusion ends.

回転可動カメラによるリアルタイム人物位置推定システム

This paper proposed a method in which location of a person is estimation with one active camera in real-time. The approach taken in this paper is as follows. First, we detect the person region with proposed method which is the region detection method based on short time distribution of the density to avoid the influence of the camera movement. Then we obtain his height and the gravity point. The rotate angle and the distance between the person and the camera are calculated by the information of the height and the gravity point. Experiments were applied with several examples. We obtained that the average error distance is less than 12[cm]. The average error distance is small enough in comparison with shoulder width of the persons. Our method can estimate the location in real-time.

Recursive mode star identification algorithms

Star identification can be accomplished by several different available algorithms that identify the stars observed by a star tracker. However, efficiency and reliability remain key issues and the availability of new active pixel cameras requires new approaches. Two novel algorithms for recursive mode star identification are presented here. The first approach is derived by the spherical polygon search (SP-search) algorithm, it was used to access all the cataloged stars observed by the sensor field-of-view (FOV) and recursively add/remove candidate cataloged stars according to the predicted image motion induced by camera attitude dynamics. Star identification is then accomplished by a star pattern matching technique which identifies the observed stars in the reference catalog. The second method uses star neighborhood information and a catalog neighborhood pointer matrix to access the star catalog. In the recursive star identification process, and under the assumption of "slow" attitude dynamics, only the stars in the neighborhood of previously identified stars are considered for star identification in the succeeding frames. Numerical tests are performed to validate the absolute and relative efficiency of the proposed methods.

Development of a Robust Photogrammetric Metrology System for Monitoring the Healing of Bedsores

AbstractA three‐camera close range photogrammetric system for robust and precise measurement of bedsores has been designed and constructed. MEDPHOS (MEDical PHOtogrammetric System) consists of three synchronised cameras with convergent optical axes. A light projector is fixed in the centre of the rig that holds the cameras. A special dot pattern is projected onto the surface to be measured, to compensate for the lack of natural texture on the wound surface. The proposed algorithm consists of the following steps: the cameras and projector are calibrated so that all interior and exterior parameters are known; tailored image segmentation procedures are developed and applied for the detection of the projected pattern dots from the uneven background of the images using morphologic operators; and watershed transformation is used to tackle the problem of overlapping pattern dots. To reduce the effects of non‐uniform illumination and specular reflection of light due to humidity (often the case with wounds), a homomorphic transformation is developed and applied to the images. After segmentation of the images, a connected‐component labelling procedure is used to establish the points for matching. The centroids of these components were precisely calculated. Intensity‐based image matching has been tested without yielding satisfactory results due to the significant deviation from the Lambertian reflection assumption used for solving the correspondence problem. This problem is reliably solved by developing a new algorithm based on geometric constraints that allow feature‐based matching and do not need approximate values of the location of the targets in the images. This robust three‐focal constraint is found to be very effective for matching provided the necessary conditions for the system configuration are met. Auxiliary photometric constraints together with the calibrated projector (which is treated like an active camera) also serve as additional sources of information for reducing the number of remaining ambiguities and checking the consistency of the results. Almost all of the required biometric information can be obtained rapidly, robustly and easily using MEDPHOS. Experimental results showed the effectiveness of the proposed technique.

Camera cooperation for achieving visual attention

In this paper we address the problem of establishing a computational model for visual attention using cooperation between two cameras. More specifically we wish to maintain a visual event within the field of view of a rotating and zooming camera through the understanding and modeling of the geometric and kinematic coupling between a static camera and an active camera. The static camera has a wide field of view thus allowing panoramic surveillance at low resolution. High-resolution details may be captured by a second camera, provided that it looks in the right direction. We derive an algebraic formulation for the coupling between the two cameras and we specify the practical conditions yielding a unique solution. We describe a method for separating a foreground event (such as a moving object) from its background while the camera rotates. A set of outdoor experiments shows the two-camera system in operation.

Smart video surveillance: exploring the concept of multiscale spatiotemporal tracking

Situation awareness is the key to security. Awareness requires information that spans multiple scales of space and time. Smart video surveillance systems are capable of enhancing situational awareness across multiple scales of space and time. However, at the present time, the component technologies are evolving in isolation. To provide comprehensive, nonintrusive situation awareness, it is imperative to address the challenge of multiscale, spatiotemporal tracking. This article explores the concepts of multiscale spatiotemporal tracking through the use of real-time video analysis, active cameras, multiple object models, and long-term pattern analysis to provide comprehensive situation awareness.

A network of co-operative cameras for visual surveillance

A network of co-operative cameras for the visual surveillance of parking lots is presented. Such a network employs multiple subnets able to manage static and active cameras in a hierarchical framework. The system is able to track multiple targets simultaneously and in real-time throughout the controlled areas. The positions of detected objects, computed from different sensors, are fused considering a dynamic reliability factor for each sensor reading. Close-up recordings of suspicious events are obtained by tasking the active camera systems (ACSs). The co-operation is performed through a multicast communication system studied to transmit useful data both intra and inter networks. In particular, information about the position of the object to track, sent by a static camera system (SCS), is used by an ACS to operate an initial repositioning. The ACS compensates background changes owing to the camera motion, detects mobile objects in the scene and autonomously tracks the object of interest. Tracking results are presented in the context of a video surveillance application for a parking lot.

1P2-S-067 PaLM-treeによる能動カメラ制御の高性能化(進化・学習とロボティクス3,生活を支援するロボメカ技術のメガインテグレーション)

1P2-S-067 PaLM-treeによる能動カメラ制御の高性能化(進化・学習とロボティクス3,生活を支援するロボメカ技術のメガインテグレーション)

Comparative Study of Human and Robotic Camera Control in Laparoscopic Biliary and Colon Surgery

Despite the growing clinical use of active robotic camera holders there is still a lack of clinical feasibility studies. We compared the use of a voice controlled robotic camera holder (AESOP 3000, Computer Motion, Goleta, California) to a human camera holder in a series of laparoscopic cholecystectomies and colectomies. Compliance with AESOP, abnormal operative events or complications, operative time, and the duration of hospitalization were prospectively recorded and compared to data recorded before the introduction of the robotic system. Compliance with AESOP was good. There were no abnormal operative events, no differences in operative time, complications, or the mean duration of hospitalization between the patients operated with a robotic or a human camera holder. The use of a robotic camera holder does not alter the length of the operative procedure, the duration of hospitalization, or postoperative morbidity. It is a safe and feasible approach to laparoscopic cholecystectomy or colectomy.