Distributed Smart Camera Networks Research Articles

Static, Dynamic, and Adaptive Heterogeneity in Distributed Smart Camera Networks

We study heterogeneity among nodes in self-organizing smart camera networks, which use strategies based on social and economic knowledge to target communication activity efficiently. We compare homogeneous configurations, when cameras use the same strategy, with heterogeneous configurations, when cameras use different strategies. Our first contribution is to establish that static heterogeneity leads to new outcomes that are more efficient than those possible with homogeneity. Next, two forms of dynamic heterogeneity are investigated: nonadaptive mixed strategies and adaptive strategies, which learn online. Our second contribution is to show that mixed strategies offer Pareto efficiency consistently comparable with the most efficient static heterogeneous configurations. Since the particular configuration required for high Pareto efficiency in a scenario will not be known in advance, our third contribution is to show how decentralized online learning can lead to more efficient outcomes than the homogeneous case. In some cases, outcomes from online learning were more efficient than all other evaluated configuration types. Our fourth contribution is to show that online learning typically leads to outcomes more evenly spread over the objective space. Our results provide insight into the relationship between static, dynamic, and adaptive heterogeneity, suggesting that all have a key role in achieving efficient self-organization.

Efficient network clustering for traffic reduction in embedded smart camera networks

In this work, a clustering approach for bandwidth reduction in distributed smart camera networks is presented. Properties of the environment such as camera positions and environment pathways, as well as dynamics and features of targets are used to limit the flood of messages in the network. To better understand the correlation between camera positioning and pathways in the scene on one hand and temporal and spatial properties of targets on the other hand, and to devise a sound messaging infrastructure, a unifying probabilistic modeling for object association across multiple cameras with disjointed view is used. Communication is efficiently handled using a task-oriented node clustering that partition the network in different groups according to the pathway among cameras, and the appearance and temporal behavior of targets. We propose a novel asynchronous event exchange strategy to handle sporadic messages generated by non-frequent tasks in a distributed tracking application. Using a Xilinx-FPGA with embedded Microblaze processor, we could show that, with limited resource and speed, the embedded processor was able to sustain a high communication load, while performing complex image processing computations.

Camera selection for tracking in distributed smart camera networks

Tracking persons with multiple cameras with overlapping fields of view instead of with one camera leads to more robust decisions. However, operating multiple cameras instead of one requires more processing power and communication bandwidth, which are limited resources in practical networks. When the fields of view of different cameras overlap, not all cameras are equally needed for localizing a tracking target. When only a selected set of cameras do processing and transmit data to track the target, a substantial saving of resources is achieved. The recent introduction of smart cameras with on-board image processing and communication hardware makes such a distributed implementation of tracking feasible. We present a novel framework for selecting cameras to track people in a distributed smart camera network that is based on generalized information-theory. By quantifying the contribution of one or more cameras to the tracking task, the limited network resources can be allocated appropriately, such that the best possible tracking performance is achieved. With the proposed method, we dynamically assign a subset of all available cameras to each target and track it in difficult circumstances of occlusions and limited fields of view with the same accuracy as when using all cameras.

Socio-economic vision graph generation and handover in distributed smart camera networks

In this article we present an approach to object tracking handover in a network of smart cameras, based on self-interested autonomous agents, which exchange responsibility for tracking objects in a market mechanism, in order to maximise their own utility. A novel ant-colony inspired mechanism is used to learn the vision graph, that is, the camera neighbourhood relations, during runtime, which may then be used to optimise communication between cameras. The key benefits of our completely decentralised approach are on the one hand generating the vision graph online, enabling efficient deployment in unknown scenarios and camera network topologies, and on the other hand relying only on local information, increasing the robustness of the system. Since our market-based approach does not rely on a priori topology information, the need for any multicamera calibration can be avoided. We have evaluated our approach both in a simulation study and in network of real distributed smart cameras.

Collaborative 3D Target Tracking in Distributed Smart Camera Networks for Wide-Area Surveillance

With the evolution and fusion of wireless sensor network and embedded camera technologies, distributed smart camera networks have emerged as a new class of systems for wide-area surveillance applications. Wireless networks, however, introduce a number of constraints to the system that need to be considered, notably the communication bandwidth constraints. Existing approaches for target tracking using a camera network typically utilize target handover mechanisms between cameras, or combine results from 2D trackers in each camera into 3D target estimation. Such approaches suffer from scale selection, target rotation, and occlusion, drawbacks typically associated with 2D tracking. In this paper, we present an approach for tracking multiple targets directly in 3D space using a network of smart cameras. The approach employs multi-view histograms to characterize targets in 3D space using color and texture as the visual features. The visual features from each camera along with the target models are used in a probabilistic tracker to estimate the target state. We introduce four variations of our base tracker that incur different computational and communication costs on each node and result in different tracking accuracy. We demonstrate the effectiveness of our proposed trackers by comparing their performance to a 3D tracker that fuses the results of independent 2D trackers. We also present performance analysis of the base tracker along Quality-of-Service (QoS) and Quality-of-Information (QoI) metrics, and study QoS vs. QoI trade-offs between the proposed tracker variations. Finally, we demonstrate our tracker in a real-life scenario using a camera network deployed in a building.

QTRAJECTORIES: IMPROVING THE QUALITY OF OBJECT TRACKING USING SELF-ORGANIZING CAMERA NETWORKS

Abstract. Previous work in the research field of video surveillance intensively focused on separated aspects of object detection, data association, pattern recognition and system design. In contrast, we propose a holistic approach for object tracking in a self-organizing and distributed smart camera network. Each observation task is represented by a software-agent which improves the tracking performance by collaborative behavior. An object tracking agent detects persons in a video stream and associates them with a trajectory. The pattern recognition agent analyses these trajectories by detecting points of interest within the observation field. These are characterized by a non-deterministic behavior of the moving person. The trajectory points (enriched by the results of the pattern recognition agent) will be used by a configuration agent to align the cameras field of view. We show that this collaboration improves the performance of the observation system by increasing the amount of detected trajectory points by 22%.

CameraNets: coverage and data management problems in distributed smart camera networks

Abstract Distributed Smart Camera systems (DSCs) consist of a (possibly large) number of cameras that collaborate on a monitoring task. DSCs have a wide range of applications such as surveillance, intelligent traffic systems, environmental monitoring, industrial safety and law enforcement. DSCs automatically control what to monitor and how to act on the collected video. For example, cameras monitoring traffic may change their orientation to track moving traffic and alert responders if an accident occurs. DSCs differ from conventional multi-camera surveillance systems in that they eliminate the need for a human to control them and to interpret the video. Free of this limitation, DSCs can scale to much higher scales, while improving monitoring effectiveness. However, a number of difficult challenges must be solved before DSCs can realize this potential. In this paper, we discuss our results and activities within a QNRF-funded project looking at two general challenges facing DSCs: 1 - Coverage control: how to control cameras with Pan-Tilt-Zoom capability to track a group of targets and/or areas of interest. We frame the problem as an optimization problem to maximize the value of the covered targets. We show that the problem is NP-hard and develop a family of heuristic approaches with near optimal behavior that do not require central coordination. 2 - Data Management: as the cameras collect their video, they need to relay it for real-time monitoring using a bandwidth constrained network, or store it for later analysis. Cameras can coordinate to eliminate redundancies and to infer the importance of the observed video. Moreover, storage architectures are needed to effectively store the video data and to allow efficient indexing and retrieval. We report our initial solutions and performance evaluation studies in this area, which were obtained by a mixture of simulation and using an experimental multi-camera test bed that we have started to deploy.

3-D Target-Based Distributed Smart Camera Network Localization

For distributed smart camera networks to perform vision-based tasks such as subject recognition and tracking, every camera's position and orientation relative to a single 3-D coordinate frame must be accurately determined. In this paper, we present a new camera network localization solution that requires successively showing a 3-D feature point-rich target to all cameras, then using the known geometry of a 3-D target, cameras estimate and decompose projection matrices to compute their position and orientation relative to the coordinatization of the 3-D target's feature points. As each 3-D target position establishes a distinct coordinate frame, cameras that view more than one 3-D target position compute translations and rotations relating different positions' coordinate frames and share the transform data with neighbors to facilitate realignment of all cameras to a single coordinate frame. Compared to other localization solutions that use opportunistically found visual data, our solution is more suitable to battery-powered, processing-constrained camera networks because it requires communication only to determine simultaneous target viewings and for passing transform data. Additionally, our solution requires only pairwise view overlaps of sufficient size to see the 3-D target and detect its feature points, while also giving camera positions in meaningful units. We evaluate our algorithm in both real and simulated smart camera networks. In the real network, position error is less than 1 ('') when the 3-D target's feature points fill only 2.9% of the frame area.

An automatic calibration method for stereo-based 3D distributed smart camera networks

Stereo-based 3D distributed smart camera networks are useful in a broad range of applications. Knowledge of the relative locations and orientations of nodes in the network is an essential prerequisite for true 3D sensing. A novel spatial calibration method for a network of pre-calibrated stereo smart cameras is presented, which obtains pose estimates suitable for collaborative 3D vision in a distributed fashion using two stages of registration on robust 3D point sets. The method is initially described in a geometrical sense, then presented in a practical implementation using existing vision and registration algorithms. Experiments using both software simulations and physical devices are designed and executed to demonstrate performance.

Optimal Camera Network Configurations for Visual Tagging

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Proper placement of cameras in a distributed smart camera network is an important design problem. Not only does it determine the coverage of the surveillance, but it also has a direct impact on the appearance of objects in the cameras which dictates the performance of all subsequent computer vision tasks. In this paper, we propose a generic camera placement model based on the visibility of objects at different cameras. Our motivation stems from the need of identifying and locating objects with distinctive visual features or “tags.” This is a very common goal in computer vision with applications ranging from identifying soccer players by their jersey numbers to locating and recognizing faces of individuals. Our proposed framework places no restriction on the visual classification tasks. It incorporates realistic camera models, self occlusion of tags, and occlusion by other moving objects. It is also flexible enough to handle arbitrary-shaped three-dimensional environments. Using this framework, two novel binary integer programming (BIP) algorithms are proposed to find the optimal camera placement for “visual tagging” and a greedy implementation is developed to cope with the complexity of BIP. Extensive performance analysis is performed using Monte Carlo simulations, virtual environment simulations, and real-world experiments. We also demonstrate the usefulness of visual tagging through robust individual identification and obfuscation across multiple camera views for privacy protection. </para>