Automatic Scene Analysis Research Articles

Parametric Synthetic Aperture Radar Image Recovery for Multiple Linear Structures: An Image Domain Approach

The linear structures in synthetic aperture radar (SAR) images can provide important geometric information regarding the illuminated objects. However, the linear structures of various objects sometimes disappear in traditional SAR images, which severely affects the application in automated scene analysis and information extraction techniques. This paper proposes a parametric SAR image recovery method for linear structures of extended targets. By extracting the spatial phase distribution feature in image domain, the proposed method is used to identify the endpoints among all of the scattered points in SAR images and reconstruct the disappeared linear structures based on the scattering model. The method can be generally divided into three procedures: endpoints and point targets classification, linear structures recognition, and linear structures reconstruction. In the first step, the endpoints of linear structures and point targets are classified by setting a threshold related to the spatial phase distribution feature. Afterwards, the linear structures recognition method is used to determine which two endpoints can be formed into a linear structure. Finally, the parametric scattering models are used to reconstruct the disappeared linear structures. Experiments are conducted on both computer simulations and the data that were acquired by microwave anechoic chamber experiment, tower crane radar experiment, and unmanned vehicle radar experiment in order to validate the effectiveness and robustness of the proposed method.

Real-Time FPGA-Based Object Tracker with Automatic Pan-Tilt Features for Smart Video Surveillance Systems

The design of smart video surveillance systems is an active research field among the computer vision community because of their ability to perform automatic scene analysis by selecting and tracking the objects of interest. In this paper, we present the design and implementation of an FPGA-based standalone working prototype system for real-time tracking of an object of interest in live video streams for such systems. In addition to real-time tracking of the object of interest, the implemented system is also capable of providing purposive automatic camera movement (pan-tilt) in the direction determined by movement of the tracked object. The complete system, including camera interface, DDR2 external memory interface controller, designed object tracking VLSI architecture, camera movement controller and display interface, has been implemented on the Xilinx ML510 (Virtex-5 FX130T) FPGA Board. Our proposed, designed and implemented system robustly tracks the target object present in the scene in real time for standard PAL (720 × 576) resolution color video and automatically controls camera movement in the direction determined by the movement of the tracked object.

Fast scene analysis using vision and artificial intelligence for object prehension by an assistive robot

Robotic assistance for people affected by motor deficits is a fast growing field. In this context, two major challenges remain in terms of automated scene analysis and automated object prehension. More specifically, the most robust of current segmentation methods are still computationally intensive, preventing the automation of objects prehension from being fast enough to be considered acceptable as an everyday technical-aid. The objective of this study is to develop a fast scene analysis using vision and artificial intelligence for object prehension by an assistive robot.The solution developed in this paper aims at facilitating human-machine interaction by enabling users to easily communicate their needs to the technical aid. To achieve this, this paper proposes several novelties in three interconnected domains: scene segmentation, prehension and recognition of 3D objects. A novel technic, inspired by mechanical probing, is developed for scenes probing to detect objects. A simple, fast and effective decision tree is proposed for object prehension. Finally, the physical characteristics of the 3D objects are directly used in the neural network without using discriminants features descriptors.The results obtained in this paper have shown that scene analysis for robotic object prehension in cooperation with a user can be performed with effective promptness. Indeed, the system requires on average 0.6s to analyze an object in a scene. With the JACO robotic assistance arm, the system can pick up a requested object in 15s while moving at 50mm/s, which may be greatly improved upon using a faster robot. The system performance averages 83% accuracy for object recognition and is able to use a decision tree to select a simple approach path for the robot end-effector towards a desired object. This system, in combination with an assistive robot, has great potential for providing users suffering from musculoskeletal disorders with improved autonomy and independence, and for encouraging sustained usage of this type of technical aids.

A keypoint-based method for background modeling and foreground detection using a PTZ camera

The automatic scene analysis is still a topic of great interest in computer vision due to the growing possibilities provided by the increasingly sophisticated optical cameras. The background modeling, including its initialization and its updating, is a crucial aspect that can play a main role in a wide range of application domains, such as vehicle tracking, person re-identification and object recognition. In any case, many challenges still remain partially unsolved, including camera movements (i.e., pan/tilt), scale changes (i.e., zoom-in/zoom-out) and deletion of the initial foreground elements from the background model. This paper describes a method for background modeling and foreground detection able to address all the mentioned challenges. In particular, the proposed method uses a spatio-temporal tracking of sets of keypoints to distinguish the background from the foreground. It analyses these sets by a grid strategy to estimate both camera movements and scale changes. The same sets are also used to construct a panoramic background model and to delete the possible initial foreground elements from it. Experiments carried out on some challenging videos from three different datasets (i.e., PBI, VOT and Airport MotionSeg) demonstrate the effectiveness of the method on PTZ cameras. Other videos from a further dataset (i.e., FBMS) have been used to measure the accuracy of the proposed method with respect to some key works of the current state-of-the-art. Finally, some videos from another dataset (i.e., SBI) have been used to test the method on stationary cameras.

A clinical assessment of cochlear implant recipient performance: implications for individualized map settings in specific environments.

Individual speech intelligibility was measured in quiet and noise for cochlear Implant recipients upgrading from the Freedom to the CP900 series sound processor. The postlingually deafened participants (n = 23) used either Nucleus CI24RE or CI512 cochlear implant, and currently wore a Freedom sound processor. A significant group mean improvement in speech intelligibility was found in quiet (Freiburg monosyllabic words at 50 dBSPL) and in noise (adaptive Oldenburger sentences in noise) for the two CP900 series SmartSound programs compared to the Freedom program. Further analysis was carried out on individual’s speech intelligibility outcomes in quiet and in noise. Results showed a significant improvement or decrement for some recipients when upgrading to the new programs. To further increase speech intelligibility outcomes when upgrading, an enhanced upgrade procedure is proposed that includes additional testing with different signal-processing schemes. Implications of this research are that future automated scene analysis and switching technologies could provide additional performance improvements by introducing individualized scene-dependent settings.

Crowded Scene Analysis: A Survey

Automated scene analysis has been a topic of great interest in computer vision and cognitive science. Recently, with the growth of crowd phenomena in the real world, crowded scene analysis has attracted much attention. However, the visual occlusions and ambiguities in crowded scenes, as well as the complex behaviors and scene semantics, make the analysis a challenging task. In the past few years, an increasing number of works on crowded scene analysis have been reported, covering different aspects including crowd motion pattern learning, crowd behavior and activity analysis, and anomaly detection in crowds. This paper surveys the state-of-the-art techniques on this topic. We first provide the background knowledge and the available features related to crowded scenes. Then, existing models, popular algorithms, evaluation protocols, as well as system performance are provided corresponding to different aspects of crowded scene analysis. We also outline the available datasets for performance evaluation. Finally, some research problems and promising future directions are presented with discussions.

Potentials of small, lightweight and low cost Multi-Echo Laser Scanners for detecting Grape Berries

Abstract. Mobile sensor devices offer great opportunities for automatic scene analysis and object recognition. Nowadays a new generation of ranging devices is available, like laser scanners which are small and light weighted. Concerning these improvements specific applications can be tackled. In this contribution we focus on vineyard monitoring for detecting and counting grape berries with a small, lightweight and low cost multi-echo laser scanner. Therefore a Hokuyo UTM-30LX-EW laser range finder is utilized for capturing the data in close range up to 1m. In order to process the data the following methodology is proposed: after smoothing and morphological techniques are applied on the laserscanning intensity and range images the number of visible grape berries is determined from the resulting segments. The approach performs with a detection accuracy of above 84%. The results reveal the high potential of such close range ranging devices for locating and counting grape berries. Thus, the methodology provides practical support for viticulture applications.

Evaluation of Intrinsic Image Algorithms to Detect the Shadows Cast by Static Objects Outdoors

In some automatic scene analysis applications, the presence of shadows becomes a nuisance that is necessary to deal with. As a consequence, a preliminary stage in many computer vision algorithms is to attenuate their effect. In this paper, we focus our attention on the detection of shadows cast by static objects outdoors, as the scene is viewed for extended periods of time (days, weeks) from a fixed camera and considering daylight intervals where the main source of light is the sun. In this context, we report two contributions. First, we introduce the use of synthetic images for which ground truth can be generated automatically, avoiding the tedious effort of manual annotation. Secondly, we report a novel application of the intrinsic image concept to the automatic detection of shadows cast by static objects in outdoors. We make both a quantitative and a qualitative evaluation of several algorithms based on this image representation. For the quantitative evaluation, we used the synthetic data set, while for the qualitative evaluation we used both data sets. Our experimental results show that the evaluated methods can partially solve the problem of shadow detection.

Evaluation of Intrinsic Image Algorithms to Detect the Shadows Cast by Static Objects Outdoors

In some automatic scene analysis applications, the presence of shadows becomes a nuisance that is necessary to deal with. As a consequence, a preliminary stage in many computer vision algorithms is to attenuate their effect. In this paper, we focus our attention on the detection of shadows cast by static objects outdoors, as the scene is viewed for extended periods of time (days, weeks) from a fixed camera and considering daylight intervals where the main source of light is the sun. In this context, we report two contributions. First, we introduce the use of synthetic images for which ground truth can be generated automatically, avoiding the tedious effort of manual annotation. Secondly, we report a novel application of the intrinsic image concept to the automatic detection of shadows cast by static objects in outdoors. We make both a quantitative and a qualitative evaluation of several algorithms based on this image representation. For the quantitative evaluation, we used the synthetic data set, while for the qualitative evaluation we used both data sets. Our experimental results show that the evaluated methods can partially solve the problem of shadow detection.

An Autonomous Framework to Produce and Distribute Personalized Team-Sport Video Summaries: A Basketball Case Study

Democratic and personalized production of multimedia content is a challenge that content providers will have to face in the near future. In this paper, we address this challenge by building on computer vision tools to automate the collection and distribution of audiovisual content. Especially, we proposed a complete production process of personalized video summaries in a typical application scenario, where the sensor network for media acquisition is composed of multiple cameras, which, for example, cover a basketball field. Distributed analysis and interpretation of the scene are exploited to decide what to show or not to show about the event, so as to produce a video composed of a valuable subset of the streams provided by each individual camera. Interestingly, the selection of the streams subsets to forward to each user depends on his/her individual preferences, making the process adaptive and personalized. The process involves numerous integrated technologies and methodologies, including but not limited to automatic scene analysis, camera viewpoint selection, adaptive streaming, and generation of summaries through automatic organization of stories. The proposed technology provides practical solutions to a wide range of applications, such as personalized access to local sport events through a web portal, cost-effective and fully automated production of content dedicated to small-audience, or even automatic log in of annotations.

Collaborative occupancy reasoning in visual sensor network for scalable smart video surveillance

Video surveillance is a popular consumer application that is used for various purposes such as public safety, facilities surveillance, and traffic monitoring. In a general video surveillance system, video streams from cameras are sent to a control center and operators monitor the videos. But human operator monitoring of the views every moment of every day is almost impossible; so, smart surveillance systems are required, systems that are capable of automated scene analysis. There are a number of studies to enable smart video surveillance in a multi-camera network. Most of the studies, however, treat central processing approaches in which a scene analysis is processed inside a central server domain once all available information has been collected in the server. Such approaches require tremendous efforts in building the system and, moreover, limit the scalability. To accomplish scalable smart video surveillance, an inference framework in visual sensor networks is necessary, one in which autonomous scene analysis is performed via distributed and collaborative processing among camera nodes without necessity for a high performance server. In this paper, we propose a collaborative inference framework for visual sensor networks and an efficient occupancy reasoning algorithm that is essential in smart video surveillance based on the framework. We estimate the existence probabilities for every camera and combine them using the work-tree architecture in a distributed and collaborative manner. We aim for practical smart video surveillance systems.

Audio-Visual Event Recognition in Surveillance Video Sequences

In the context of the automated surveillance field, automatic scene analysis and understanding systems typically consider only visual information, whereas other modalities, such as audio, are typically disregarded. This paper presents a new method able to integrate audio and visual information for scene analysis in a typical surveillance scenario, using only one camera and one monaural microphone. Visual information is analyzed by a standard visual background/foreground (BG/FG) modelling module, enhanced with a novelty detection stage and coupled with an audio BG/FG modelling scheme. These processes permit one to detect separate audio and visual patterns representing unusual unimodal events in a scene. The integration of audio and visual data is subsequently performed by exploiting the concept of synchrony between such events. The audio-visual (AV) association is carried out online and without need for training sequences, and is actually based on the computation of a characteristic feature called audio-video concurrence matrix, allowing one to detect and segment AV events, as well as to discriminate between them. Experimental tests involving classification and clustering of events show all the potentialities of the proposed approach, also in comparison with the results obtained by employing the single modalities and without considering the synchrony issue

Error analysis of robust optical flow estimation by least median of squares methods for the varying illumination model

The apparent pixel motion in an image sequence, called optical flow, is a useful primitive for automatic scene analysis and various other applications of computer vision. In general, however, the optical flow estimation suffers from two significant problems: the problem of illumination that varies with time and the problem of motion discontinuities induced by objects moving with respect to either other objects or with respect to the background. Various integrated approaches for solving these two problems simultaneously have been proposed. Of these, those that are based on the LMedS (Least Median of Squares) appear to be the most robust. The goal of this paper is to carry out an error analysis of two different LMedS-based approaches, one based on the standard LMedS regression and the other using a modification thereof as proposed by us recently. While it is to be expected that the estimation accuracy of any approach would decrease with increasing levels of noise, for LMedS-like methods, it is not always clear as to how much of that decrease in performance can be attributed to the fact that only a small number of randomly selected samples is used for forming temporary solutions. To answer this question, our study here includes a baseline implementation in which all of the image data is used for forming motion estimates. We then compare the estimation errors of the two LMedS-based methods with the baseline implementation. Our error analysis demonstrates that, for the case of Gaussian noise, our modified LMedS approach yields better estimates at moderate levels of noise, but is outperformed by the standard LMedS method as the level of noise increases. For the case of salt-and-pepper noise, the modified LMedS method consistently performs better than the standard LMedS method.

A comparison of algorithms for inference and learning in probabilistic graphical models

Research into methods for reasoning under uncertainty is currently one of the most exciting areas of artificial intelligence, largely because it has recently become possible to record, store, and process large amounts of data. While impressive achievements have been made in pattern classification problems such as handwritten character recognition, face detection, speaker identification, and prediction of gene function, it is even more exciting that researchers are on the verge of introducing systems that can perform large-scale combinatorial analyses of data, decomposing the data into interacting components. For example, computational methods for automatic scene analysis are now emerging in the computer vision community. These methods decompose an input image into its constituent objects, lighting conditions, motion patterns, etc. Two of the main challenges are finding effective representations and models in specific applications and finding efficient algorithms for inference and learning in these models. In this paper, we advocate the use of graph-based probability models and their associated inference and learning algorithms. We review exact techniques and various approximate, computationally efficient techniques, including iterated conditional modes, the expectation maximization (EM) algorithm, Gibbs sampling, the mean field method, variational techniques, structured variational techniques and the sum-product algorithm ("loopy" belief propagation). We describe how each technique can be applied in a vision model of multiple, occluding objects and contrast the behaviors and performances of the techniques using a unifying cost function, free energy.

Towards intelligent image retrieval

Research into techniques for the retrieval of images by semantic content is still in its infancy. This paper reviews recent trends in the field, distinguishing four separate lines of activity: automatic scene analysis, model-based and statistical approaches to object classification, and adaptive learning from user feedback. It compares the strengths and weaknesses of model-based and adaptive techniques, and argues that further advances in the field are likely to involve the increasing use of techniques from the field of artificial intelligence.

Vehicle guidance by automated scene analysis

Automatic guided vehicles normally require some form of fixed guidance. But in work being performed at City University guidance will be by using machine processed visual information only.

Automated scene analysis of CT scans

Since the advent of computed tomography, there has been an increasing realization that CT scans contain quantitative as well as qualitative information useful in the diagnostic process. Often however, the use of this information is impeded by the tedious manual outlining of the areas of interest in the scan. To alleviate this problem, we have developed a scene segmentation algorithm which will automatically delineate areas of interest in a CT scan. This procedure uses known information about the expected objects in the scan in conjunction with an algorithm to label those objects. The resultant scene segmentation divides the scan into four anatomical areas: skull, normal brain, high density lesions and CSF. After an area of interest is interactively selected by the clinician, volume, density or other quantitative measures may be computed. Limitations of the algorithm and its clinical applications are discussed.