Cooperative Surveillance Research Articles

Topology optimization of UAV network for target surveillance task with support jamming

In this paper, a topology optimization method is proposed for cooperative surveillance in unmanned aerial vehicles (UAVs) under an interference environment. We introduce the regional coverage rate to evaluate the cooperative surveillance performance under interference. At its core, we derive the expression of detection probability under interference in detail, which is inherently tied to the signal to interference plus noise ratio (SINR) influenced by UAV configuration, in which the location of each UAV affects the cooperative surveillance performance. To maximize the network surveillance performance, we use the coverage rate metric as the objective function to formulate a topology optimization problem under the interference environment, while satisfying distance constraints between UAVs. As the problem is non-convex and highly nonlinear, the internal self-constrained particle swarm optimization (ISC-PSO) algorithm is used to solve this problem, which can well address the issue of range constraints while maintaining adequate convergence. The numerical studies demonstrate that the proposed algorithm can obtain good surveillance performance in terms of different jamming strategies and shapes of monitoring areas.

ESIT – a digital twin of air surveillance infrastructure

Civil air traffic surveillance and management depends heavily on the use of aircraft transponders replying to radio interrogations from cooperative surveillance systems such as secondary surveillance radars, multilateration and collision avoidance systems. A high number of such surveillance systems combined with a high aircraft density can cause a large quantity of interrogations, therefore having a detrimental impact on transponder behavior and sometimes even causing a loss of replies and thus a deterioration in aircraft surveillance. In order to ensure the safe operation of European air traffic management, the European Commission obligates the member states to take measures to ensure that the transponder load in their air spaces does not exceed a safe level. EUROCONTROL, in its function as network manager responsible for supporting the monitoring, management and coordination of the scarce spectrum resource, has contracted the implementation of a European simulation tool to Austro Control in collaboration with SeRo Systems and the Institute of Microwave and Photonic Engineering as subcontractors. This paper examines the development and use of this tool as well as the functionality it provides.

Dynamic Perimeter Surveillance of Multiagent Systems: A Swarm-Based Approach.

In this article, a novel approach to generating kinematic state trajectories for single-integrator multiagent systems is developed with the aim of addressing cooperative surveillance tasks of rectangular areas. In particular, the key idea consists in driving the involved agents within a so-called containment region while simultaneously reaming outside a forbidden area around the prescribed target. As one of its main features, the proposed kinematics allows the whole multiagent configuration to rotate safely along the perimeter under both full and partial connectivity properties of the underlying topology graph. Steady-state conditions are analyzed and sufficient conditions are derived in terms of kinematic model parameters. Finally, a set of simulations is aimed at showing the capability of the kinematic descriptions to quickly buttonhole the containment region and to keep a rotating behavior around the target.

Evaluation of the quality of radar information of dependent cooperative surveillance systems

Evaluation of the quality of radar information of dependent cooperative surveillance systems

Information Entropy-Based Cooperative Surveillance in Areas with Obstacles

This paper solves the problem of searching a surveillance region and tracking possible radio emitters by multiple unmanned aerial vehicles cooperating through communication links in a decentralized manner. An innovative entropy-based approach is utilized to obtain the routes and tasks (search/track) for the vehicles. Another significant contribution of this research is the assumption of each vehicle being equipped with only one simple and cost-effective direction finding (DF) sensor. The routes and tasks are determined by minimizing a well-defined entropy-based cost function. The proposed surveillance scheme allows one to find a feasible and near-optimal solution. The algorithm’s efficiency is verified through various simulations.

Global benefits and domestic costs of a cooperative surveillance strategy to control transboundary crop pathogens

Societal Impact StatementSurveillance of plant pathogens is usually designed according to country boundaries. Benefits of a global surveillance system to tackle long‐distance dispersed crop pathogens are unquantified. Here, a ‘non‐cooperative’ and a ‘cooperative’ strategy are compared in terms of minimizing the surveillance effort to achieve given domestic and global targets. Although a ‘cooperative’ strategy is always more suitable, impacts of its adoption are not equally distributed among countries. Medium‐sized countries in central Europe and Asia would benefit the most from reducing the domestic effort, whereas others would need to deploy more sentinels than they would place in their own interests.Summary Transboundary diseases are extremely complex to control and can cause global socio‐economic damage. In the context of crop protection, surveillance strategies are usually designed according to country boundaries, regardless of the spatial scale of the spread of the disease. In this study, we investigate the suitability of this scale for surveilling long‐distance dispersed pathogens. We use an epidemic network describing worldwide potential transport of Puccinia graminis, the causal agent of stem rust of wheat, modelled in a previous work. Based on network properties, we conceive two strategies for prioritizing areas to be monitored for the presence of the disease, either cooperative or each country alone, and we compare their performances in terms of minimizing the effort deployed in achieving given surveillance targets at global and domestic level. We find that a cooperative strategy is more efficient at the global scale. However, its adoption implies a heterogeneous geographic distribution of surveillance effort‐related costs and benefits. Medium‐sized countries in central Europe and Asia would benefit the most; on the other hand, countries placed in important spreading pathways should deploy more surveillance effort than they would place without cooperation. Among the major wheat producers, China is the only country that may have a cost from a cooperative strategy, whereas India, Russia, the United States, France and Ukraine would have the most benefits. The acknowledgement of how costs and benefits of a global governance would be shared among countries is needed to gain unanimous support for an international cooperative surveillance system.

SMART: Vision-Based Method of Cooperative Surveillance and Tracking by Multiple UAVs in the Urban Environment

UAV surveillance and tracking have attracted great enthusiasm in intelligent transportation, and various approaches have been reported up to now. However, these approaches often ignored the uncertainties in the urban environment, such as occlusion, view change, and background clutter. Ignoring these uncertain factors often leads to a reduction in surveillance performance and tracking quality. This study devotes to improving the cooperative surveillance capability of multi-UAV formation by designing different cooperative strategies in the urban environment. To be specific, a novel cooperative architecture is designed to control the observation locations of multiple UAVs throughout the formation process. For different types of interference, we introduce a novel target recognition rate of each UAV as the decision factor and design corresponding cooperative strategies to guarantee the accuracy of cooperative surveillance. Based on this architecture, we develop a vision-based method of cooperative surveillance and tracking by multiple UAVs (SMART) whose objective function is the motion cost and flight reliability of UAVs to ensure that each UAV can be in the optimal surveillance location for the target. The proposed SMART skillfully integrates the strict, elastic, and flight constraint strategies. During the execution of the multi-UAV formation, the inherent safety constraints of multiple UAVs and the designed strategies are used to solve the quadratic optimization model to adjust the locations of these UAVs. To demonstrate the superiority of our method, we conduct a 3D simulation urban environment and devise several experiments to analyze the performance of SMART on it. The experimental results demonstrate that SMART can not only maintain the high cooperative flight capability, but also provide high flexibility and fault tolerance.

Cooperative Path Optimization for Multiple UAVs Surveillance in Uncertain Environment

Research on multiple unmanned aerial vehicles (UAVs) cooperative surveillance systems serving Internet of Things (IoT) applications, such as smart cities, precision logistics, etc., has become a hot topic. However, the target movement is unpredictable in an uncertain environment, and multiple UAVs are affected by obstacles or inaccessible regions, resulting in the decreased surveillance performance and even the loss of the target. This article is dedicated to determine the current surveillance environment through the 2-D laser scanner. At the cost of the energy consumption and the transmission unreliability, a multi-UAV cooperative path optimization (MCPO) model is designed to adjust the surveillance location of each UAV, which improves the target surveillance performance. Specifically, for different types of obstacles or inaccessible regions, we present a novel obstacle-avoidance selection strategy with two mechanisms in mind: 1) when some of UAVs encounter obstacles, but others can accurately monitor the target, a strict constraint mechanism is established to promptly adjust the surveillance location of each UAV, which ensures the accuracy of formation surveillance and 2) when all UAVs have to avoid obstacles, a fuzzy constraint mechanism is presented and combined with Lucas–Kanade (LK) method to expand the search range of the multi-UAV and enhance the flexible adjustment capability of the formation. To verify the superiority of the proposed optimization method, we develop a 3-D simulation experiment environment based on the UE4 platform and design several groups of experiments to analyze the effectiveness of MCPO. The experimental results demonstrate that MCPO can not only maintain the flight stability of multiple UAVs but also has satisfactory formation flexibility and surveillance accuracy.

Multi-Target Association for UAVs Based on Triangular Topological Sequence

Multi-UAV cooperative systems are highly regarded in the field of cooperative multi-target localization and tracking due to their advantages of wide coverage and multi-dimensional perception. However, due to the similarity of target visual characteristics and the limitation of UAV sensor resolution, it is difficult for UAVs to correctly distinguish targets that are visually similar to their associations. Incorrect correlation matching between targets will result in incorrect localization and tracking of multiple targets by multiple UAVs. In order to solve the association problem of targets with similar visual characteristics and reduce the localization and tracking errors caused by target association errors, based on the relative positions of the targets, the paper proposes a globally consistent target association algorithm for multiple UAV vision sensors based on triangular topological sequences. In contrast to Siamese neural networks and trajectory correlation, the relative position relationship between targets is used to distinguish and correlate targets with similar visual features and trajectories. The sequence of neighboring triangles of targets is constructed using the relative position relationship, and the feature is a specific triangular network. Moreover, a method for calculating topological sequence similarity with similar transformation invariance is proposed, as well as a two-step optimal association method that considers global objective association consistency. The results of flight experiments indicate that the algorithm achieves an association accuracy of 84.63%, and that two-step association is 12.83% more accurate than single-step association. Through this work, the multi-target association problem with similar or even identical visual characteristics can be solved in the task of cooperative surveillance and tracking of suspicious vehicles on the ground by multiple UAVs.

Evaluation of the Interference’s Impact of Cooperative Surveillance Systems Signals Processing for Healthcare

Patient signals produced from a physical device, such as electrocardiography (ECG), which records the electrical activity of the heart, are vulnerable to keep noise due to various physical constraints of acquisition devices. It is critical for the detection and diagnosis of a variety of diseases. An ECG signal should be displayed as clean and clear as feasible due to its relevance in assisting physicians and doctors in making appropriate judgments. ECG is vulnerable to many types of noise because it is an electrical signal. To improve the quality of assistance for consumers Info a surveillance system by cooperative surveillance systems, high-quality data processing by the observed surveillance systems is required, which predetermines the requirement for high noise immunity of the latter. At the same time, the basics of building cooperative surveillance systems as a network of two-channel asynchronous information communication systems which include numeral of transmitting and receiving process using diverse frequency limits for reception and transmission, failure-prone open single-channel queuing systems, and request signals do not allow to provide the required the noise resistance of the systems under consideration. This paper first gives a characterization of request and interference signal flows in cooperative surveillance systems and briefly examines the features of request signals and their effect on the immunity of cooperative surveillance systems. Then, it calculates the noise resistance of the demand signals of the cooperative surveillance systems for the probabilities of errors: signal skipping; First- and second-degree false alarms in the case of unrelated events, unintended and intra-system impulse interferences in the request channel

SurveilNet: A Lightweight Anomaly Detection System for Cooperative IoT Surveillance Networks

The boring and repetitive task of monitoring video feeds makes real-time anomaly detection tasks difficult for humans. Hence, crimes are usually detected hours or days after the occurrence. To mitigate this, the research community proposes the use of a deep learning-based anomaly detection model (ADM) for automating the monitoring process. However, the isolated setup of existing surveillance systems makes ADM inefficient and susceptible to staleness due to the lack of resource sharing and continuous learning (CL). CL is the incremental development of models that adapts continuously to the external world. Thus, for efficient CL in surveillance systems, devices must share resources and cooperate with neighbor sites. Yet, solutions from the literature focus on the isolated environment thereby neglecting the need for resource sharing and CL. To address this gap, this paper proposes a cooperative surveillance system called SurveilNet that allows for resource sharing between surveillance sites under the control of a cooperator node. We further propose a lightweight subscription scheme that allows for a joint specialized model development process that continually adapts to the dynamics of the secured environment. Our proposed scheme offers the ability to learn from the neighboring site’s data without compromising data privacy. The performance of our scheme is evaluated using a reclassified UCF-Crime dataset with the result showing the efficiency of our proposed scheme when compared to the state-of-the-art.

UAV-UGV-UMV Multi-Swarms for Cooperative Surveillance.

In this paper we present a surveillance system for early detection of escapers from a restricted area based on a new swarming mobility model called CROMM-MS (Chaotic Rössler Mobility Model for Multi-Swarms). CROMM-MS is designed for controlling the trajectories of heterogeneous multi-swarms of aerial, ground and marine unmanned vehicles with important features such as prioritising early detections and success rate. A new Competitive Coevolutionary Genetic Algorithm (CompCGA) is proposed to optimise the vehicles’ parameters and escapers’ evasion ability using a predator-prey approach. Our results show that CROMM-MS is not only viable for surveillance tasks but also that its results are competitive in regard to the state-of-the-art approaches.

Cooperative Control-Based Task Assignments for Multiagent Systems With Intermittent Communication

Efficient task assignments can significantly improve agent management and reduce communication load and energy consumption. This article investigates the cooperative control problem for multiagent systems with an active task assignment strategy, in which whether an agent exchanges the information with neighbors depends on a perceived mission. By defining a set of missions, a task assignment mechanism for cooperative control problem is first proposed, in which the tasks will be scheduled by intermittent communication signals associated with the actual optimization requirements. By allowing appropriate task assignment conditions, a class of tracking cooperative control protocol is designed and accordingly, the stability of the closed-loop systems under the intermittent communication will be guaranteed. We also consider a case that the communication links between the followers and the leader can be optimized. To maximize the quality of information interaction, a leadership competition mechanism is introduced to design the tracking cooperative control protocol. As an application, cooperative surveillance using a group of rotary-wing air vehicles is considered. Numerical simulation demonstrates the effectiveness of the proposed approaches.

MCDP: Maximizing Cooperative Detection Probability for Barrier Coverage in Rechargeable Wireless Sensor Networks

Barrier coverage problem is an important issue in wireless sensor networks (WSNs). Many solutions have been proposed for constructing a defense barrier aiming to maximize the surveillance quality while prolonging the network lifetime. However, most of them assumed that the sensor nodes are battery powered without considering the rechargeable sensors. Though the energy conservation issue has been taken into account in most studies, the perpetual network lifetime is still impossible. In addition, most of existing works considered Boolean Sensing Model (BSM) which cannot reflect the physical features of sensing. This paper proposes a barrier coverage algorithm, called MCDP, which considers the rechargeable solar-powered sensors and applies the Probability Sensing Model (PSM) aiming to maximize the surveillance quality while the perpetual network lifetime of WSNs can be achieved. The MCDP algorithm first partitions the time line and monitoring region into several space time points and calculates the detection probability of each sensor to each space time point. Then the proposed MCDP algorithm schedules sensors staying in sensing state and recharging state in each time slot such that the weakest cooperative surveillance quality of the time-space point can be maximized. Experimental study shows that the proposed MCDP algorithm achieves better performance than existing work in terms of surveillance quality, stability of cooperative detection probability as well as fault tolerance.

Adaptive level of autonomy for human-UAVs collaborative surveillance using situated fuzzy cognitive maps

Collaborating with a squad of Unmanned Aerial Vehicles (UAVs) is challenging for a human operator in a cooperative surveillance task. In this paper, we propose a cognitive model that can dynamically adjust the Levels of Autonomy (LOA) of the human-UAVs team according to the changes in task complexity and human cognitive states. Specifically, we use the Situated Fuzzy Cognitive Map (SiFCM) to model the relations among tasks, situations, human states and LOA. A recurrent structure has been used to learn the strategy of adjusting the LOA, while the collaboration task is separated into a perception routine and a control routine. Experiment results have shown that the workload of the human operator is well balanced with the task efficiency.

Complex Field Network Coding for Multi-Source Multi-Relay Single-Destination UAV Cooperative Surveillance Networks

Relay-based cooperative communication for unmanned aerial vehicle (UAV) networks can obtain spatial diversity gains, expand coverage, and potentially increase the network capacity. A multi-source multi-relay single-destination structure is the main topology structure for UAV cooperative surveillance networks, which is similar to the structure of network coding (NC). Compared with conventional NC schemes, complex field network coding (CFNC) can achieve a higher throughput and is introduced to surveillance networks in this paper. According to whether there is a direct communication link between any source drone and the destination, the information transfer mechanism at the downlink is set to one of two modes, either mixed or relay transmission, and two corresponding irregular topology structures for CFNC-based networks are proposed. Theoretical analysis and simulation results with an additive white Gaussian noise (AWGN) channel show that the CFNC obtains a throughput as high as 1/2 symbol per source per channel use. Moreover, the CFNC applied to the proposed irregular structures under the two transmission modes can achieve better reliability due to full diversity gain as compared to that based on the regular structure. Moreover, the reliability of the CFNC scheme can continue to be improved by combining channel coding and modulation techniques at the expense of rate loss.

Probabilistic Safety Assessment for UAS Separation Assurance and Collision Avoidance Systems

The airworthiness certification of aerospace cyber-physical systems traditionally relies on the probabilistic safety assessment as a standard engineering methodology to quantify the potential risks associated with faults in system components. This paper presents and discusses the probabilistic safety assessment of detect and avoid (DAA) systems relying on multiple cooperative and non-cooperative tracking technologies to identify the risk of collision of unmanned aircraft systems (UAS) with other flight vehicles. In particular, fault tree analysis (FTA) is utilized to measure the overall system unavailability for each basic component failure. Considering the inter-dependencies of navigation and surveillance systems, the common cause failure (CCF)-beta model is applied to calculate the system risk associated with common failures. Additionally, an importance analysis is conducted to quantify the safety measures and identify the most significant component failures. Results indicate that the failure in traffic detection by cooperative surveillance systems contribute more to the overall DAA system functionality and that the probability of failure for ownship locatability in cooperative surveillance is greater than its traffic detection function. Although all the sensors individually yield 99.9% operational availability, the implementation of adequate multi-sensor DAA system relying on both cooperative and non-cooperative technologies is shown to be necessary to achieve the desired levels of safety in all possible encounters. These results strongly support the adoption of a unified analytical framework for cooperative/non-cooperative UAS DAA and elicits an evolution of the current certification framework to properly account for artificial intelligence and machine-learning based systems.

Fuzzy multiobjective cooperative surveillance of multiple UAVs based on distributed predictive control for unknown ground moving target in urban environment

In this paper, a fuzzy multiobjective path planning method based on distributed predictive control is proposed to deal with the problem of cooperative searching and tracking of unknown ground moving target by multiple unmanned aerial vehicles (UAVs) in urban environment. Firstly, extended Kalman filter (EKF) is combined with probability estimation to predict the states of the unknown target. Secondly, the line of sight occlusion of buildings, and energy consumptions of UAVs and sensors are considered in path planning. The objective functions are designed as target coverage degree, control input cost of UAV and sensor energy consumption respectively. The cooperative surveillance path planning problem is transformed into multiobjective optimization with different importance levels. Thirdly, distributed predictive control is used to obtain the local optimal path of each UAV. The predictive states of UAVs in finite horizon are exchanged to build the collision avoidance constraint, and the minimum turning radius constraint is also addressed. Then, all the objectives are fuzzified to handle the different importance level requirement. The sensor energy consumption function with switch value is equivalently converted using Sigmoid function and sign function. According to the principle that the objective with higher priority has higher satisfactory degree, preemptive priorities are transformed into the relaxed order of satisfactory degrees. The best path satisfying the requirement of multiobjective optimization and importance levels can be obtained. Finally, the simulation results show the effectiveness of the proposed method by comparing with traditional multiobjective weighted algorithm.

Energy efficient data aggregation and improved prediction in cooperative surveillance system through Machine Learning and Particle Swarm based Optimization

The present pandemic demands touchless and autonomous, intelligent surveillance system to reduce human involvement. Heterogeneous types of sensors are used to improve the effectiveness of this surveillance system and a cooperative approach of such sensors will make the system

Cooperative Vehicular Traffic Monitoring in Realistic Low Penetration Scenarios: The COLOMBO Experience.

The relevance of effective and efficient solutions for vehicle traffic surveillance is widely recognized in order to enable advanced strategies for traffic management, e.g., based on dynamically adaptive and decentralized traffic light management. However, most related solutions in the literature, based on the powerful enabler of cooperative vehicular communications, assume the complete penetration rate of connectivity/communication technologies (and willingness to participate in the collaborative surveillance service) over the targeted vehicle population, thus making them not applicable nowadays. The paper originally proposes an innovative solution for cooperative traffic surveillance based on vehicular communications capable of: (i) working with low penetration rates of the proposed technology and (ii) of collecting a large set of monitoring data about vehicle mobility in targeted areas of interest. The paper presents insights and lessons learnt from the design and implementation work of the proposed solution. Moreover, it reports extensive performance evaluation results collected on realistic simulation scenarios based on the usage of iTETRIS with real traces of vehicular traffic of the city of Bologna. The reported results show the capability of our proposal to consistently estimate the real vehicular traffic even with low penetration rates of our solution (only 10%).