Global Camera Research Articles

Compressed domain indexing of scalable H.264/SVC streams

We present methods to efficiently analyze scalable, compressed H.264/scalable video coding (SVC) video streams. Relying solely on information present in the compressed stream, we estimate the global camera motion, perform motion segmentation and use a simple matching process to track moving objects over time. Object energy images are constructed in order to help resolve the problem of object correspondence during the occlusions of multiple objects. To save computing time, we analyze lower spatial layers of the stream and add higher layer information only if necessary. We draw 2-D object trajectories in the view plane of the camera and use the temporal evolution of the objects’ properties to estimate the relative distance to the camera, resulting in a pseudo 3-D representation of the trajectories. Finally, the suitability of the motion parameters to perform video retrieval/copy detection tasks is demonstrated. We therefore form two simple descriptors that are invariant to a series of transformations.

Real-Time Digital Image Stabilization System Using Modified Proportional Integrated Controller

This paper presents a novel, real-time digital image stabilization (DIS) system using a modified proportional integrated (MPI) controller to remove stably unwanted shaking from an image sequence that is captured by a hand-held video camera without affecting the deliberate panning motion of the camera. Researchers have addressed the trade-off problem between the removal of shaking and the preservation of global camera movement. This paper analyzes motion compensation, which is formulated as a control problem. A compensating motion vector (CMV) estimation method with a modified PI control system is proposed both to remove the unwanted jitter and to preserve the deliberate, panning motion of the camera. Experimental results demonstrate that the proposed system provides robust motion compensation of image sequences in various conditions.

The Autonomous Golf Playing Micro Robot: With Global Vision and Fuzzy Logic Controller

Abstract This paper presents design concepts in developing an Autonomous Golf Playing Micro Robot. The robot moves autonomously using 2 DC motors controlled by a remote PC using fuzzy logic system. A servomotor is used for the motion of the putter in hitting the golf ball. A microcontroller is used to process the information sent by the computer through RF communication and drive the robot motors for appropriate motions. The robot plays golf in a defined environment and avoids obstacles. A real golf ball with orange color is use during the game. The system utilizes global camera vision that serves as the eyes of the robot to know its dynamic environment for navigation. The vision system developed detects the position of the robot, golf hole, golf boundaries, and the golf ball within the playing field on real time using color object recognition algorithm. A modified golf tournament between autonomous robot golf player and man operated robot golf player was conducted. Results shows the accuracy and robustness of the autonomous robot developed in performing its tasks.

Detailed Real-Time Urban 3D Reconstruction from Video

The paper presents a system for automatic, geo-registered, real-time 3D reconstruction from video of urban scenes. The system collects video streams, as well as GPS and inertia measurements in order to place the reconstructed models in geo-registered coordinates. It is designed using current state of the art real-time modules for all processing steps. It employs commodity graphics hardware and standard CPU's to achieve real-time performance. We present the main considerations in designing the system and the steps of the processing pipeline. Our system extends existing algorithms to meet the robustness and variability necessary to operate out of the lab. To account for the large dynamic range of outdoor videos the processing pipeline estimates global camera gain changes in the feature tracking stage and efficiently compensates for these in stereo estimation without impacting the real-time performance. The required accuracy for many applications is achieved with a two-step stereo reconstruction process exploiting the redundancy across frames. We show results on real video sequences comprising hundreds of thousands of frames.

Video frame rate up conversion under inconsistent camera motion

In this paper, we address the problem of video frame rate up-conversion (FRC) in the compressed domain. FRC is often recognized as video temporal interpolation. This problem is very challenging when targeted for video sequences with inconsistent camera and object motion, such as sports videos. A novel compressed domain motion compensation scheme is presented and applied in this paper, aiming at up-sampling frame rates in sports videos. MPEG-2 encoded motion vectors (MVs) are utilized as inputs in the proposed algorithm. The decoded MVs undergo a cumulative spatiotemporal interpolation. An iterative rejection scheme based on the dense motion vector field (MVF) and the generalized affine motion model is exploited to detect global camera motion. Subsequently, the foreground object separation is performed by additionally examining the temporal consistency of the output of iterative rejections. This consistency check process helps coalesce the resulting foreground blocks and weed out the unqualified blocks. Finally, different compensation strategies for the camera and object motions are applied to interpolate the new frames. Illustrative examples are provided to demonstrate the efficacy of the proposed approach. Experimental results are compared with the popular block and non-block based frame interpolation approaches.

Application of Super-Resolution Image Reconstruction to Digital Holography

We describe a new application of super-resolution image reconstruction to digital holography which is a technique for three-dimensional information recording and reconstruction. Digital holography has suffered from the low resolution of CCD sensors, which significantly limits the size of objects that can be recorded. The existing solution to this problem is to use optics to bandlimit the object to be recorded, which can cause the loss of details. Here super-resolution image reconstruction is proposed to be applied in enhancing the spatial resolution of digital holograms. By introducing a global camera translation before sampling, a high-resolution hologram can be reconstructed from a set of undersampled hologram images. This permits the recording of larger objects and reduces the distance between the object and the hologram. Practical results from real and simulated holograms are presented to demonstrate the feasibility of the proposed technique.

Self-Localization for Indoor Mobile Robots Using Multiple Optical Mouse Sensor Values and Simple Global Camera Information

This paper describes a self-localization for indoor mobile robots based on integrating measurement values from multiple optical mouse sensors and a global camera. This paper consists of two parts. Firstly, we propose a dead-reckoning based on increments of the robot movements read directly from the floor using optical mouse sensors. Since the measurement values from multiple optical mouse sensors are compared to each other and only the reliable values are selected, accurate dead-reckoning can be realized compared with the conventional method based on increments of wheel rotations. Secondly, in order to realize robust localization, we propose a method of estimating position and orientation by integrating measured robot position (orientation information is not included) via global camera and dead-reckoning with the Kalman Filter.

Motion-based motion deblurring

Motion blur due to camera motion can significantly degrade the quality of an image. Since the path of the camera motion can be arbitrary, deblurring of motion blurred images is a hard problem. Previous methods to deal with this problem have included blind restoration of motion blurred images, optical correction using stabilized lenses, and special cmos sensors that limit the exposure time in the presence of motion. In this paper, we exploit the fundamental trade off between spatial resolution and temporal resolution to construct a hybrid camera that can measure its own motion during image integration. The acquired motion information is used to compute a point spread function (psf) that represents the path of the camera during integration. This psf is then used to deblur the image. To verify the feasibility of hybrid imaging for motion deblurring, we have implemented a prototype hybrid camera. This prototype system was evaluated in different indoor and outdoor scenes using long exposures and complex camera motion paths. The results show that, with minimal resources, hybrid imaging outperforms previous approaches to the motion blur problem. We conclude with a brief discussion on how our ideas can be extended beyond the case of global camera motion to the case where individual objects in the scene move with different velocities.

MPEG-2 압축 영역에서 움직이는 객체의 추적 및 해석

본 논문에서는 MPEG-2비디오 스트림에서 복호화 과정 없이 압축비디오에서 직접 얻을 수 있는 정보들을 활용하여 움직이는 객체를 추적하고 해석하는 방법을 제안한다. 제안된 방법에서는 먼저 MPEG-2의 움직임 벡터로부터 근사적으로 움직임 플로우(motion new)를 구성하고, 전역 적인 움직임 플로우로부터 일반화된 Hough 변환을 이용 카메라의 기본적인 움직임인 팬(pan), 틸트(tilt), 줌(zoom)량 등을 계산하였다. 계산된 카메라 움직임은 국부적으로 일어나는 객체의 움직임을 보정하는데 사용하였다. 움직이는 객체의 추적은 사용자가 원하는 객체를 바운딩 박스 형태로 정의함으로 시동된다. 이후의 객체의 추적은 카메라 움직임이 보정된 객체의 움직임 플로우를 한 GOP(Group of Pictures)단위로 면적 기여도에 따라 누적하여 추적하였다. 또한 추적오차의 누적을 막기 위해 매 GOP마다 DCT(Discrete Cosine Transform) 정보를 이용하여 초기 바운딩 박스와 매칭을 통해 객체의 영역을 재 설정하였다. 제안된 방법은 압축된 비디오 스트림에서 직접 정보를 얻음으로써 계산속도의 향상을 기할 수 있으나, 압축된 MPEG-2 비디오에서 얻을 수 있는 정보들이 최대 블록 단위이므로 객체의 정의도 블록단위 이상의 객체로 제한되며, 이용한 수 있는 정보가 제한되어 있기 때문에 정확한 객체추적보다는 근사적인 객체추적에 적합하다. This paper proposes a method to trace and interpret a moving object based on the information which can be directly obtained from MPEG-2 compressed video stream without decoding process. In the proposed method, the motion flow is constructed from the motion vectors included in compressed video. We calculate the amount of pan, tilt, and zoom associated with camera operations using generalized Hough transform. The local object motion can be extracted from the motion flow after the compensation with the parameters related to the global camera motion. Initially, a moving object to be traced is designated by user via bounding box. After then automatic tracking Is performed based on the accumulated motion flows according to the area contributions. Also, in order to reduce the cumulative tracking error, the object area is reshaped in the first I-frame of a GOP by matching the DCT coefficients. The proposed method can improve the computation speed because the information can be directly obtained from the MPEG-2 compressed video, but the object boundary is limited by macro-blocks rather than pixels. Also, the proposed method is proper for approximate object tracking rather than accurate tracing of an object because of limited information available in the compressed video data.

Developing a Barrier-Free Map by Using Global Positioning System and Cellular Phones with Digital Cameras

The recent enactment of the transportation barrier-free law in Japan emphasizes the importance of incorporating a barrier-free approach for any new road facility construction and for improvements to existing road facilities in urban areas. The purpose of this study is to develop barrierfree maps for people with various types of disabilities to help public agencies plan a barrier-free transportation system environment. Statistics show that technological progress and the explosive proliferation of cellular phone use in Japan has reached the level of two phones for every three people. In particular, new advanced cellular phones equipped with additional features such as a Global Positioning System (GPS) and digital cameras are gaining in popularity. The focus is on GPS cellular phones with digital cameras as a tool to collect barrier-free information on the streets. The digital photos and location information generated by the GPS are sent to a personal computer by e-mail. The gathered information is then stored in a geographic information system (GIS) database to display hazardous barrier locations with pictures and messages on a map. Actual field surveys were conducted in the central business district of Utsunomiya City using this method, and information was collected about various barriers for people with different types of disabilities. Barrier-free maps for different types of disabilities were then developed using the multilayer feature of GIS. By comparing hazardous locations for each group of disabilities, both common and differential characteristics were found and summarized in the form of a barrier-free map. Such barrier-free maps are expected to be useful to public agencies for providing a barrier-free transportation environment for people with all types of disabilities.

Image sequence stabilisation using fuzzy adaptive Kalman filtering

Image sequence stabilisation based on fuzzy adaptive Kalman filtering of absolute frame displacements is proposed. Two Kalman filters are operated in parallel, one of which is used as a reference filter and is run with a constant high process noise variance to enable smooth and close tracking of global camera displacements. The process noise variance of the stabilisation filter is adaptively varied according to the residual between the stabilisation filter and the reference filter output.

Real-Time Digital Image Stabilization Using Kalman Filters

This paper presents a novel, real-time stabilization system that uses Kalman filters to remove short-term image fluctuations with retained smooth gross movements. The global camera motion is defined in terms of constant acceleration motion and constant velocity motion models, and Kalman filtering is employed to facilitate smooth operation. It is shown that the process noise variance has a direct effect on stabilization performance, and that it is possible to implement an efficient and robust stabilization system by adaptively changing the process noise variance value according to long-term camera motion dynamics.

Flame recognition in video

This paper presents an automatic system for fire detection in video sequences. There are several previous methods to detect fire, however, all except two use spectroscopy or particle sensors. The two that use visual information suffer from the inability to cope with a moving camera or a moving scene. One of these is not able to work on general data, such as movie sequences. The other is too simplistic and unrestrictive in determining what is considered fire; so that it can be used reliably only in aircraft dry bays. We propose a system that uses color and motion information computed from video sequences to locate fire. This is done by first using an approach that is based upon creating a Gaussian-smoothed color histogram to detect the fire-colored pixels, and then using a temporal variation of pixels to determine which of these pixels are actually fire pixels. Next, some spurious fire pixels are automatically removed using an erode operation, and some missing fire pixels are found using region growing method. Unlike the two previous vision-based methods for fire detection, our method is applicable to more areas because of its insensitivity to camera motion. Two specific applications not possible with previous algorithms are the recognition of fire in the presence of global camera motion or scene motion and the recognition of fire in movies for possible use in an automatic rating system. We show that our method works in a variety of conditions, and that it can automatically determine when it has insufficient information.

Retrieval of video clips using global motion information

A new scheme is proposed to extract a global motion model from a general video sequence. Since global camera motion often conveys the semantic meaning of a video's content, these global model parameters are used as features for video retrieval. Experimental results show that the scheme is quite effective in locating video segments with similar motion content in football game sequences.

Image sequence stabilisation based on DFT filtering

An image sequence stabilisation system based on DFT filtering of absolute frame displacements to compensate for undesired fluctuations in the sequence by shifting image frames into correct positions is reported. The system compensates for undesired jitter, while preserving desired global camera motions. Robustness is introduced to interframe motion estimation by averaging motion vectors detected in the phase correlation surface.

Scintillation Camera Field Uniformity: Visual or Quantitative?

Global and local scintillation camera uniformity were studied phenomenolgically with emphasis on statistical features. A system parameter (the systematic error) was developed to quantitate global field uniformity. (The systematic error also appears to predict the maximum variation in response resulting from local irradiation of the detector.) Uniformity correction by a statistically weighted matrix renormalization procedure was demonstrated to improve the global frequency distribution of the counts per pixel. Such a renormalization procedure also slightly improved local response if the systematic error was small but not if the systematic error was large. The failure of this renormalization procedure appeared to correlate with long range interactions in the detector. The existence of such long range interactions casts doubt on the efficacy (and even the desirability) of many commonly used uniformity corrections.

Electric field observations by incoherent scatter radar in the auroral zone

Analysis of ion drift velocity measurements, made for a 24-hour period in February 1972 by the incoherent scatter radar at Chatanika, Alaska (L = 5.7), provides a detailed view of electric fields and currents in the auroral zone. Large northward electric fields were seen in the evening sector, and an abrupt change to southward occurred at the start of a midnight sector substorm. The westward electric field was generally much smaller than the north-south electric field but showed a rapid increase at the time of a westward auroral surge. Study of global magnetograms and all-sky camera photographs has led to the identification of five substorms during the 24-hour observation period. These substorms were sufficiently separated in time to allow identification of the electric field and current variations in late afternoon, evening, and morning local time sectors.