Visual Sensor Networks Research Articles

Visual Sensors Hardware Platforms: A Review

In the last two decades, visual sensor networks have been developed to gather visual data from different monitoring scenarios, directly contributing to the maturation of the Internet of Things (IoT) applications landscape. Such developments have evolved according to the releasing of new hardware platforms for the construction of sensor nodes, as well as the availability of low-power compatible cameras. Since sensor networks deployed for visual data sensing are inherently complex and resource-demanding, knowing how visual sensors have been created can give a better perception of how those networks have evolved and which innovations are expected for the next years. This article reviews past developments of visual sensors concerning the employed hardware platforms and cameras, providing a taxonomy for them and discussing how to choose the most appropriate hardware when designing and implementing modern (wireless) visual sensor networks.

Unraveling Impact of Critical Sensing Range on Mobile Camera Sensor Networks

In camera sensor networks (CSNs), full view coverage, meaning that any direction of any point in the operational region is covered by at least one camera sensor, plays a significant role in object identification. While prior work is dedicated to static CSNs for the sake of critical condition to achieve full view coverage, such performance still remains unknown in mobile CSNs. In this paper, we take the initiative to address this issue, where a centralized parameter, i.e., equivalent sensing radius (ESR), is defined to unravel the critical requirement for asymptotic full view coverage in mobile heterogeneous CSNs in the sense that camera sensors of different sensing capabilities are moving around in target area. Specifically, we derive ESR under three different mobilities, i.e., 1-dimensional and 2-dimensional random walks and random rotating model, and then explore respectively the corresponding critical conditions to achieve almost surely coverage. 1 The static network is introduced as a baseline in order to gain a clear understanding of how mobility affects coverage performance differently. Interestingly, we find that both 1-dimensional and 2-dimensional random walks exhibit a smaller ESR than static one whereas ESR is even larger in random rotating mobility than that in static CSNs. Moreover, the almost surely coverage is found to be around 1.225 times of the critical condition to achieve coverage with high probability, 2 and therefore turns out to be a stronger result compared to the traditional coverage with high probability. We then turn to the impact of various mobility patterns on sensing energy consumption, a metric that is closely related to ESR, and show that it can be decreased by random walks under certain delay tolerance. The relationship between ESR and percentage of full view coverage is also discussed and the results unify those under homogeneous CSNs.

Accurate and Flexible Calibration Method for a Class of Visual Sensor Networks

Accurate and flexible calibration is a prerequisite for visual sensor networks to retrieve metric information from image data. This paper presents the design and implementation of an accurate and flexible calibration method for a class of visual sensor networks intended for 3D measurement and tracking in large volumes. The proposed method employs a generic camera model that is applicable for wide-angle lens cameras as well as for conventional cameras. It does not require all the employed cameras to share a common field of view, and only pairwise overlap is needed. In the calibration process, the poses between stereo cameras are first initialized using essential matrix decomposition and then optimized by the Levenberg-Marquardt algorithm. A weighted vision graph is proposed to select optimal transformation paths among cameras by using Dijkstra's shortest path algorithms for multi-camera calibration. Then, the global coordinates are constructed using a four-marker calibration triangle. Finally, a Unity3D-based virtual platform, in which the total number and configurations of cameras, as well as the environment scene, can be arbitrarily edited, is designed to test the proposed calibration algorithms. Extensive experiments based on synthetic and real data are performed to demonstrate the effectiveness of the proposed multi-camera calibration method. Experimental results show that the multi-camera calibration method is accurate and easy-to-implement in the presence of noise.

Dynamic Node Cooperation Based on Moving Multi-target Tracking in Double Wireless Video Sensor Network

Dynamic Node Cooperation Based on Moving Multi-target Tracking in Double Wireless Video Sensor Network

Mask-Refined R-CNN: A Network for Refining Object Details in Instance Segmentation.

With the rapid development of flexible vision sensors and visual sensor networks, computer vision tasks, such as object detection and tracking, are entering a new phase. Accordingly, the more challenging comprehensive task, including instance segmentation, can develop rapidly. Most state-of-the-art network frameworks, for instance, segmentation, are based on Mask R-CNN (mask region-convolutional neural network). However, the experimental results confirm that Mask R-CNN does not always successfully predict instance details. The scale-invariant fully convolutional network structure of Mask R-CNN ignores the difference in spatial information between receptive fields of different sizes. A large-scale receptive field focuses more on detailed information, whereas a small-scale receptive field focuses more on semantic information. So the network cannot consider the relationship between the pixels at the object edge, and these pixels will be misclassified. To overcome this problem, Mask-Refined R-CNN (MR R-CNN) is proposed, in which the stride of ROIAlign (region of interest align) is adjusted. In addition, the original fully convolutional layer is replaced with a new semantic segmentation layer that realizes feature fusion by constructing a feature pyramid network and summing the forward and backward transmissions of feature maps of the same resolution. The segmentation accuracy is substantially improved by combining the feature layers that focus on the global and detailed information. The experimental results on the COCO (Common Objects in Context) and Cityscapes datasets demonstrate that the segmentation accuracy of MR R-CNN is about 2% higher than that of Mask R-CNN using the same backbone. The average precision of large instances reaches 56.6%, which is higher than those of all state-of-the-art methods. In addition, the proposed method requires low time cost and is easily implemented. The experiments on the Cityscapes dataset also prove that the proposed method has great generalization ability.

Study of Energy Management in Wireless Visual Sensor Networks

Wireless Visual Sensor Networks (WVSNs) are a branch of Wireless Sensor Networks (WSNs), WVSN nodes vary from standard WSN nodes in the ability of sensing the environment in two dimensions rather than in one. Therefore, it follows the three main fundamentals of WSNs: wireless networking, distributed sensing and low power hardware. This paper discusses different challenges that face the design of WVSNs like deployment of nodes, field of view overlapping, image analysis, area coverage and energy consumption. Efforts have been done mainly to survey the problem of energy consumption that can affect the lifetime of visual sensor network and overview the different techniques that have been used by many researchers to handle this crucial issue.

Face Detection and Recognition for use in Campus Surveillance

The study of image processing in today’s world and the booming possibility of building a smart classroom and a smarter campus with the help of aided vigilance and surveillance is slowly moving from a thought that can be considered for the future to an actual real-world implementation. In modern day schools and university campuses there is an increasing demand for a real-time monitoring and quick responding database that tracks the student activities. This is not always required but serves as a one-click system that handles average information searches and returns the list of fast action options that are available in tracking the campus activities that come in the purview of its span. In a diverse educational campus comprising of several branches and streams that share a single campus, there is a possible chance of intrusion and unauthorized entry which may lead to undesirable and unnecessary loss of intellectual property and manipulation of identity. In a particular academic unit, there would be a surveillance system that monitors and tracks these activities and offers to its privileged users a response in real-time. It can also be used to track the attendance of students in an automated fashion which leads to a digitized and paperless approach. It can be said that this will be implemented in its entirety to a campus unit. Face identification is an essential step in face recognition, in which one of the typical of a class and authoritative application in visual sensor network. Visual perception is one of the physical measurements based on secured features. Face identification is a demanding assignment, because it has to scan and match against a library of known faces. E.g. lighting condition, different posture, various kind of body languages.

3D camera sensor scheduling algorithms for indoor multi-objective tracking

With the advantages of sensing data’s diversity, camera sensor networks (CSNs) have been applied to objective tracking in both outdoor and indoor environments. In indoor objective tracking, the objectives’ moving mode and sensors’ deployment have limitation and particularity, which bring more challenges on persistent monitoring for CSNs. In this paper, we consider the indoor multi-objective tracking in three-dimensional (3D) CSNs and focus on the 3D camera sensor scheduling for objective tracking to improve the coverage quality of the objective trajectory and minimizing the whole working periods. We firstly introduce the active-period-minimizing scheduling problem in CSNs for indoor objective tracking, with the goal of minimizing the total active periods of sensors. We solve the problem via three algorithms: the first algorithm is designed based on our proposed projection-based algorithm for single-objective case; the second one is proposed with the main idea of path coloring and the third one is a divide-and-conquer strategy with an approximation ratio of $$H(\frac{area(\mathcal {B})}{gside^2})$$. To evaluate these algorithms’ performance on the time efficiency, we conduct extensive simulation experiments and analyze their results on the time efficiency advantages and applicable scenarios.

Communication and computation inter-effects in people counting using intelligence partitioning

The rapid development of the Internet of Things is affecting the requirements towards wireless vision sensor networks (WVSN). Future smart camera architectures require battery-operated devices to facilitate deployment for scenarios such as industrial monitoring, environmental monitoring and smart city, consequently imposing constraints on the node energy consumption. This paper provides an analysis of the inter-effects between computation and communication energy for a smart camera node. Based on a people counting scenario, we evaluate the trade-off for the node energy consumption with different processing configurations of the image processing tasks, and several communication technologies. The results indicate that the optimal partition between the smart camera node and remote processing is with background modelling, segmentation, morphology and binary compression implemented in the smart camera, supported by Bluetooth Low Energy (BLE) version 5 technologies. The comparative assessment of these results with other implementation scenarios underlines the energy efficiency of this approach. This work changes pre-conceptions regarding design space exploration in WVSN, motivating further investigation regarding the inclusion of intermediate processing layers between the node and the cloud to interlace low-power configurations of communication and processing architectures.

An anomaly detection method using deep convolution neural network for vision image of robot

With the acceleration of urbanization, growing number of places are crowded with people, such as banks, shopping malls, schools and hospitals, and the incidence of abnormal events such as assault, fighting, trampling and evacuation is also increasing. Therefore, the need for intelligent detection and identification of abnormal events by security early warning robots is attracting much more attention. Aiming at the problem of anomaly detection for security early warning robots, an anomaly detection method using wireless vision sensor network (WVSN) and deep learning is proposed. Firstly, image collection is carried out by WVSN, and video image information in the monitoring range is transmitted and stored by WVSN. Then, the collected image is preprocessed, and the possible abnormal areas are effectively extracted by region of interest (ROI), image filtering and region segmentation. Finally, the abnormal areas are extracted by WVSN. The slow feature analysis (SFA) is used to solve the problem of insufficient training samples in the deep neural network. Furthermore, the deep convolution neural network (CNN) and the support vector machine (SVM) are used to train and complete the classification respectively. The experimental results on UMN and PETS 2009 database show that the abnormal events can be effectively detected by the proposed method. Compared with several other advanced methods, the proposed method has higher detection accuracy and area under the curve (AUC). Among them, AUC on the experimental data set can reach up to 0.998. Therefore, the proposed method has a good reference value for the application of security early warning robots in densely populated places.

Energy-efficient face detection and recognition scheme for wireless visual sensor networks

Energy-efficient and robust face detection and recognition scheme can be useful for many application fields such as security and surveillance in multimedia and visual sensor network (VSN). VSN consists of wireless resources-constrained nodes that are equipped with low-energy CMOS cameras for monitoring. On the one hand, captured images are meaningful multimedia-data that impose high energy consumption to be processed and transmitted. On the other hand, visual sensor (VS) is a battery-powered node with limited life-time. This situation leads to a trade-off between detection-accuracy and power-consumption. This trade-off is considered as the most major challenge for applications using multimedia data in wireless environments such as VSN. For optimizing this trade-off, a novel face detection and recognition scheme has been proposed in this paper based on VSN. In this scheme, detection phase is performed at VS and recognition phase is accomplished at the base station (sink). The contributions of this paper are in three folds: 1. Fast and energy-aware face-detection algorithm is proposed based on omitting non-human blobs and feature-based face detection in the considered human-blobs. 2. A novel energy-aware and secure algorithm for extracting light-weight discriminative vector of detected face-sequence to be sent to sink with low transmission-cost and high security level. 3. An efficient face recognition algorithm has been performed on the received vectors at the sink. The performance of our proposed scheme has been evaluated in terms of energy-consumption, detection and recognition accuracy. Experimental results, performed on standard datasets (FERET, Yale and CDnet) and on personal datasets, demonstrate the superiority of our scheme over the recent state-of-the-art methods.

Wireless Visual Sensor Networks Energy Optimization Based on New Entropy Model

Last advances in low power sensors has led to the development of wireless visual sensor networks. These networks comparing to traditional wireless sensor networks provides valuable visual information, therefor are suitable for surveillance and control applications. One of the important challenges in wireless visual sensor networks is energy consumption. Therefore, in this paper we address the problem of energy optimization in wireless visual sensor networks. The entropy of the captured images in each camera node sensor as a quality criteria is used. For realization of this goal, we introduced a new formula for expressing the entropy of captured image in each camera node. This model is a function of the distance between camera node and target and the angel between the main view line of camera and the target. Then, we introduced a new formula which expresses the relation between the image entropy and the number of bits required for displaying image pixel value. In next step, we proposed node selection algorithm based on entropy. For proposing our algorithm, first optimization of energy consumption is formulated, then the problem is solved as a convex problem. We used from CVX library in MATLAB and Log-Barrier Method for solving our problem and shown the simulation result of them. Finally, we compare our proposed algorithm with MDT benchmark algorithm, CVX and Log-Barrier algorithms. Also we compare the sensitivity to error for both algorithms.

A Comprehensive Data Gathering Network Architecture in Large-Scale Visual Sensor Networks.

The fundamental utility of the Large-Scale Visual Sensor Networks (LVSNs) is to monitor specified events and to transmit the detected information back to the sink for achieving the data aggregation purpose. However, the events of interest are usually not uniformly distributed but frequently detected in certain regions in real-world applications. It implies that when the events frequently picked up by the sensors in the same region, the transmission load of LVSNs is unbalanced and potentially cause the energy hole problem. To overcome this kind of problem for network lifetime, a Comprehensive Visual Data Gathering Network Architecture (CDNA), which is the first comparatively integrated architecture for LVSNs is designed in this paper. In CDNA, a novel α-hull based event location algorithm, which is oriented from the geometric model of α-hull, is designed for accurately and efficiently detect the location of the event. In addition, the Chi-Square distribution event-driven gradient deployment method is proposed to reduce the unbalanced energy consumption for alleviating energy hole problem. Moreover, an energy hole repairing method containing an efficient data gathering tree and a movement algorithm is proposed to ensure the efficiency of transmitting and solving the energy hole problem. Simulations are made for examining the performance of the proposed architecture. The simulation results indicate that the performance of CDNA is better than the previous algorithms in the realistic LVSN environment, such as the significant improvement of the network lifetime.

Priority-based Routing Framework for Image Transmission in Visual Sensor Networks: Experimental Analysis

A Visual Sensor Network (VSN) is a specialized Wireless Sensor Network (WSN) equipped with cameras. Its primary function is to capture images, videos and send them to power rich sink nodes for processing. As image data is much larger than scalar data sensed by a typical WSN, applications of VSN require much bigger amount of data to be transferred to the sink. Due to constraints of WSN such as low energy, limited CPU power and scarce memory, transmission of large amount of data becomes challenging. On the other hand, some VSN applications require critical image features sooner than the entire image to take action. In this paper, we provide the details of experiments done using our proposed Priority-based Routing Framework for Image Transmission (PRoFIT). PRoFIT is designed to deliver critical image features at high priority to the sink node for early processing. Peak signal-to-noise ratio (PSNR) analyses show that PRoFIT improves VSN application response time as compared to priority-less routing. This paper also contains the design of our VSN testbed. Multiple indoor and outdoor experiments were performed to validate the framework. This framework also improves the energy efficiency of the network. The results show that the PRoFIT is 40% efficient in terms of energy consumption.

Coverage dynamic optimisation for video sensor networks based on PSO algorithm

Under the deterministic deployment of the video sensor nodes, the nodes are fixed in position and adjustable in direction. In order to improve the coverage of video sensor networks, a multi-group directional sensing model is designed, and a coverage dynamic optimisation for video sensor networks based on particle swarm optimisation algorithm is proposed. The designed algorithm adopts cosine adaptive inertia weight, and adaptively adjusts it from large to small with the number of iterations. It can improve the coverage rate of networks by optimising the sensing model. The experimental results show that the proposed algorithm has better results of different sensing angles, and can improve coverage rate in the condition of deterministic deployment of nodes.

Coverage dynamic optimisation for video sensor networks based on PSO algorithm

Under the deterministic deployment of the video sensor nodes, the nodes are fixed in position and adjustable in direction. In order to improve the coverage of video sensor networks, a multi-group directional sensing model is designed, and a coverage dynamic optimisation for video sensor networks based on particle swarm optimisation algorithm is proposed. The designed algorithm adopts cosine adaptive inertia weight, and adaptively adjusts it from large to small with the number of iterations. It can improve the coverage rate of networks by optimising the sensing model. The experimental results show that the proposed algorithm has better results of different sensing angles, and can improve coverage rate in the condition of deterministic deployment of nodes.

Modelling Coverage Failures Caused by Mobile Obstacles for the Selection of Faultless Visual Nodes in Wireless Sensor Networks

Wireless sensor networks comprising nodes equipped with cameras have become common in many scenarios, providing valuable visual data for some relevant services such as localization, tracking, patterns identification and emergencies detection. In this context, algorithms and optimization approaches have been designed to perform different types of quality assessment or performance enhancement tasks, addressing challenging issues such as networking, compression, availability, reliability, security, energy efficiency and virtually any subject related to the operational challenges of those networks. However, the dynamics of coverage failures have not been properly modelled in visual sensor networks, resulting in unrealistic perceptions when optimizing or assessing quality in most visual sensing scenarios. Particularly, the Field of View of visual sensors will be affected by occlusion caused by obstacles in the monitored field, which may turn such sensors inadequate for the expected monitoring services of the considered network. Therefore, this article proposes a mathematical model to assess occlusion caused by mobile obstacles such as vehicles on a road or forklifts in an industrial plant, aiming at the selection of the visual sensor nodes that will not have their coverage significantly restricted by those obstacles. Doing so, the proposed model can be exploited by any optimization or quality assessment approach in wireless visual sensor networks, providing a preprocessing method when selecting visual nodes.

Hardware-Friendly Laplacian-Based Multi-Focus Image Fusion in DCT Domain for Visual Sensor Network

Visual sensor network (VSN) requires a multi-focus image or video frame fusion technique involving focus measure computation in the DCT-domain to generate an all-in-focus image. Such techniques are implemented on resource-constrained on-board systems requiring hardware-friendly implementations. In this article, we first show that components of the Laplacian matrix are related to the discrete cosine transform (DCT) basis. The relation is that the eigenvalues of the Laplacian with proper boundary condition form the diagonal elements of the diagonal matrix generated by the DCT operation on the Laplacian. Exploiting this relation, we propose a focus measure which works on the DCT coefficients reflecting the spatial-domain Laplacian operation. Certain simplifications allow our focus measure computation through hardware-friendly integer multiplication and summation, where matrix multiplication involves just N scalar multiplications for an N × N 2D signal. Finally, we propose an approach which suitably fuses multi-focus images or video frames in DCT based image or video coding framework through detection of properly focused area and neighborhood consistency analysis. We show that our proposed approach is hardwarefriendly, computationally simple, and is fast enough for VSN. Through experimental results, we show that our approach outperforms the relevant state-of-the-art in multi-focus image fusion for VSN both quantitatively and subjectively. We also show that our approach is effective in comparison to the state-of-the-art and a few latest generic multi-focus image fusion techniques in terms of quantitative and subjective evaluations.

基于改进PSO的视频传感器网络覆盖增强算法

基于改进PSO的视频传感器网络覆盖增强算法

FoV-Based Quality Assessment and Optimization for Area Coverage in Wireless Visual Sensor Networks

Wireless visual sensor networks are commonly employed on several applications contexts such as smart cities, intelligent transportation systems and industrial management, aiming at the use of visual data from cameras to provide enhanced information and to expand the networks utilities. In these scenarios, some applications may require high-definition images when performing more specialized tasks, for example in face and text recognition, adding an important monitoring requirement when using camera-based sensors. In fact, it is important to ensure that the network is able to gather visual data with the associated required quality to each task, and such perceived quality may be processed as a function of the Field of View (FoV) of the visual sensors. In order to address this issue, new quality metrics are proposed for wireless visual sensor networks that are deployed to perform area coverage, exploiting for that different perceptions of the FoV. Those metrics are proposed along with redeployment optimization methods for visual sensor nodes aiming at the improvement of the perceived monitoring quality, which are based on greedy and evolutionary-based approaches. The proposed metrics and algorithms are expected to be more realistic than previous solutions, allowing flexible processing of variables as cameras' positions, orientations and viewing angles, providing then high flexibility on the definition of parameters and significantly contributing to the development of sensor networks based on visual sensors.