Airborne Surveillance Research Articles

Geo-registration and Geo-location Using Two Airborne Video Sensors

Geo-registration and geo-location of data collected by video sensors such as electro-optical and infrared cameras are two fundamental steps in the airborne surveillance of ground targets. With the availability of high-resolution imaging sensors and detailed mapping or terrain data sources, video data plays an increasingly important role in modern surveillance platforms like unmanned aerial vehicles and airborne, ground, or maritime surveillance systems. Surveillance systems without any compensation for the inevitable sensor registration errors, i.e., biases, may make geo-location erroneous and render the surveillance platform less effective for precision targeting. This article deals with the modeling of sensor biases in geo-location and proposes a method to estimate them. The proposed method leads to a minimization problem with a nonlinear cost function. Detailed derivation of the bias model is given along with an algorithm to find the bias parameters. The achievable lower bounds for debiased geo-location are provided and simulations are used to demonstrate the validity of the proposed method.

Space-Time Adaptive Processing Techniques (STAP) for Mitigation of Jammer Interference and Clutter Suppression in Airborne Radar Systems: A MATLAB Implementation-based Study

Radar Systems are used for the detection of objects by transmitting the electromagnetic waves in the free space. They operate in environments where there is a high possibility that the desired echo signal interferes with the signals from other sources. These signals include clutters and jammer signals. The jammer is a device that continuously emits the wideband radio signals in the radar environment, to saturate the receiver with noise or false information. Thus, the total received signal has three components- returns from target, clutter, and jammer combined i.e. it is a three dimensional signal. The use of conventional signal processing techniques is not desirable, as they cannot separate the desired echo signal from the other components, because the statistics of these components present in the received signal is not known. This problem needs to be accounted for, in airborne surveillance radars, as they have to identify and locate the targets in multiple interference environments. The Space-Time Adaptive Techniques (STAP) is a combination of spatial and temporal filtering that can nullify the jammer signal, and recognize the slow-moving targets. These techniques filter the signal in the angular and the Doppler domain for suppressing the unwanted signals. This paper presents a theoretical study of space-time adaptive coding techniques, and the MATLAB implementation of STAP algorithms; namely, SMI, DPCA, and ADPCA, to suppress clutter and jammer interference in the received pulse.

India's eye in the sky

Two former defence research scientists and a journalist narrate a gripping story of how India found her way into the exclusive club of nations with airborne surveillance technology. Two former defence research scientists and a journalist narrate a gripping story of how India found her way into the exclusive club of nations with airborne surveillance technology.

Interview

interview

Electrooptical complex for terrain on-time survey

The article deals with a number of approaches to the use of electrooptical systems and unmanned aerial vehicles (UAV) to solve the tasks of live terrain surface screening, air incident spots and objects survey. The paper provides the grounds for the UAV airframe chosen structure and the design of the electrooptical complex. The light weight, high -wing monoplane airframe structure is suggested. Using the CNC production center, a UAV with high aerodynamic performance was developed. The UAV was fitted with the optoelectronic system. Structural design for the airborne and ground equipment of the survey complex was developed. The structural design comprises the forward-looking and side airborne surveillance cameras and also downward-looking cameras of visible and infrared bands to provide the day and night vision. The downward-looking cameras of visible and infrared bands, integrated into the UAV fuselage belly are used for the surface monitoring and survey. The flight test on the use of electrooptic system comprised the flight on the route and automatic return using the satellite system navigation. The test showed that navigation data correction was required. The positioning accumulative error elimination is suggested to be done via automated image registration with the contour extraction and landmark reference. The image processing results using Canny algorithm were presented. The recommendations on the algorithm practical application and speed of response are given. To solve the problem of terrain on-time surveillance and monitoring the flight routes were subjected to research. Based on the experiments performed, the recommendations were given on the electrooptic complex application and flight route plotting for the tasks of on-time search and rescue.

Gunshot Airborne Surveillance with Rotary Wing UAV-Embedded Microphone Array.

Unmanned aerial vehicles (UAV) are growing in popularity, and recent technological advances are fostering the development of new applications for these devices. This paper discusses the use of aerial drones as a platform for deploying a gunshot surveillance system based on an array of microphones. Notwithstanding the difficulties associated with the inherent additive noise from the rotating propellers, this application brings an important advantage: the possibility of estimating the shooter position solely based on the muzzle blast sound, with the support of a digital map of the terrain. This work focuses on direction-of-arrival (DoA) estimation methods applied to audio signals obtained from a microphone array aboard a flying drone. We investigate preprocessing and different DoA estimation techniques in order to obtain the setup that performs better for the application at hand. We use a combination of simulated and actual gunshot signals recorded using a microphone array mounted on a UAV. One of the key insights resulting from the field recordings is the importance of drone positioning, whereby all gunshots recorded in a region outside a cone open from the gun muzzle presented a hit rate close to 96%. Based on experimental results, we claim that reliable bearing estimates can be achieved using a microphone array mounted on a drone.

Meet the First Authors.

Meet the First Authors.

Graceful Degradation An Airborne Surveillance Radar Perspective

Active electronically scanned antenna (AESA)-based radars imbibe the desirable feature of ‘graceful degradation’. Such radars use miniaturised transmit-receive (TR) modules and a failure of few modules does not lead to failure of the mission. For example, in AESA-based ground MTI radar, failure of a few modules does not affect the array performance. In such a case, the static ground clutter is centred on zero frequency does not have a motion dependent Doppler spread. However, in airborne AESA radars, the ground clutter has an angle dependent Doppler frequency due to the platform motion and clutter leaking in through antenna side-lobes. Hence, the antenna side lobe levels dictate the side lobe clutter against which target detection is to be performed. The detection performance is governed by the signal to interference plus noise ratio (SINR). For Airborne surveillance radar the effect of random and systematic failures of TR modules and their effect on SINR is characterised. It is shown that single channel processing does not effectively provide the graceful degradation feature as the SINR loss due to failures is significant. However, the effect of systematic failure on SINR loss is less as compared to random failures. An effective scheme for feeding the array is also proposed.

Convolutional Neural Network-Based Multi-Target Detection and Recognition Method for Unmanned Airborne Surveillance Systems

This paper proposes the convolutional neural network (CNN)-based multiple targets detection and recognition method for unmanned airborne surveillance systems. The proposed method is capable of recognizing the target’s type, position and bearing angle. Recently, deep learning approaches using convolutional neural networks (CNNs) have significantly improved the object detection accuracy on benchmark datasets such as Pascal visual object classes (VOC) and common objects in context (COCO) data sets. Typical CNN-based object detection technologies are designed to recognize regions of interest (RoI) and object classes based on VOC or COCO data set criteria only. However, in many surveillance missions, the bearing angle of the object is also an important entity to infer in addition to the RoI and the vehicle-type. This paper proposes a CNN-based object recognition technique called airborne surveillance neural network (ASNet) that can recognize this additional bearing angle information. Indoor experiments demonstrate the validity of the proposed method.

An Adaptive Framework for Multi-Vehicle Ground Speed Estimation in Airborne Videos

With the rapid development of unmanned aerial vehicles (UAVs), UAV-based intelligent airborne surveillance systems represented by real-time ground vehicle speed estimation have attracted wide attention from researchers. However, there are still many challenges in extracting speed information from UAV videos, including the dynamic moving background, small target size, complicated environment, and diverse scenes. In this paper, we propose a novel adaptive framework for multi-vehicle ground speed estimation in airborne videos. Firstly, we build a traffic dataset based on UAV. Then, we use the deep learning detection algorithm to detect the vehicle in the UAV field of view and obtain the trajectory in the image through the tracking-by-detection algorithm. Thereafter, we present a motion compensation method based on homography. This method obtains matching feature points by an optical flow method and eliminates the influence of the detected target to accurately calculate the homography matrix to determine the real motion trajectory in the current frame. Finally, vehicle speed is estimated based on the mapping relationship between the pixel distance and the actual distance. The method regards the actual size of the car as prior information and adaptively recovers the pixel scale by estimating the vehicle size in the image; it then calculates the vehicle speed. In order to evaluate the performance of the proposed system, we carry out a large number of experiments on the AirSim Simulation platform as well as real UAV aerial surveillance experiments. Through quantitative and qualitative analysis of the simulation results and real experiments, we verify that the proposed system has a unique ability to detect, track, and estimate the speed of ground vehicles simultaneously even with a single downward-looking camera. Additionally, the system can obtain effective and accurate speed estimation results, even in various complex scenes.

Knowledge-Aided Doppler Beam Sharpening Super-Resolution Imaging by Exploiting the Spatial Continuity Information.

This paper deals with the problem of high cross-range resolution Doppler beam sharpening (DBS) imaging for airborne wide-area surveillance (WAS) radar under short dwell time situations. A knowledge-aided DBS (KA-DBS) imaging algorithm is proposed. In the proposed KA-DBS framework, the DBS imaging model for WAS radar is constructed and the cross-range resolution is analyzed. Since the radar illuminates the imaging scene continuously through the scanning movement of the antenna, there is strong spatial coherence between adjacent pulses. Based on this fact, forward and backward pulse information can be predicted, and the equivalent number of pulses in each coherent processing interval (CPI) will be doubled based on the autoregressive (AR) technique by taking advantage of the spatial continuity property of echoes. Finally, the predicted forward and backward pulses are utilized to merge with the initial pulses, then the newly merged pulses in each CPI are utilized to perform the DBS imaging. Since the number of newly merged pulses in KA-DBS is twice larger than that in the conventional DBS algorithm with the same dwell time, the cross-range resolution in the proposed KA-DBS algorithm can be improved by a factor of two. The imaging performance assessment conducted by resorting to real airborne data set, has verified the effectiveness of the proposed algorithm.

New Saddle-Point Technique for Non-Coherent Radar Detection With Application to Correlated Targets in Uncorrelated Clutter Speckle

This paper presents a novel implementation of the saddle-point technique for calculating the non-coherent detection probability of targets by airborne surveillance radar in the presence of pulse-to-pulse correlation. It is based on the inverse Laplace transform of the moment generating function for the returned power distribution by means of integration along the path of steepest descent. To illustrate the methodology, the problem of non-coherent pulse integration for a target with gamma-fluctuating partially correlated radar cross-section embedded within compound surface clutter with uncorrelated speckle is revisited. This situation arises for maritime or overland surveillance radar with sufficient frequency agility to fully decorrelate the clutter speckle but not the fluctuations of the target radar cross-section. An exact method and a hierarchy of approximations are described, all of which perform well for this problem. Monte Carlo simulation at the quadrature component level is used to verify the results.

A Hierarchical Association Framework for Multi-Object Tracking in Airborne Videos

Multi-Object Tracking (MOT) in airborne videos is a challenging problem due to the uncertain airborne vehicle motion, vibrations of the mounted camera, unreliable detections, changes of size, appearance and motion of the moving objects and occlusions caused by the interaction between moving and static objects in the scene. To deal with these problems, this work proposes a four-stage hierarchical association framework for multiple object tracking in airborne video. The proposed framework combines Data Association-based Tracking (DAT) methods and target tracking using a compressive tracking approach, to robustly track objects in complex airborne surveillance scenes. In each association stage, different sets of tracklets and detections are associated to efficiently handle local tracklet generation, local trajectory construction, global drifting tracklet correction and global fragmented tracklet linking. Experiments with challenging airborne videos show significant tracking improvement compared to existing state-of-the-art methods.

Model-Based Forecasting of Agricultural Crop Disease Risk at the Regional Scale, Integrating Airborne Inoculum, Environmental, and Satellite-Based Monitoring Data

Crop diseases have the potential to cause devastating epidemics that threaten the world’s food supply and vary widely in their dispersal pattern, prevalence and severity. It remains unclear what the impact disease will have on sustainable crop yields in the future. Agricultural stakeholders are increasingly under pressure to adapt their decision-making to make more informed and efficient use of irrigation water, fertilizers and pesticides. They also face increasing uncertainty in how best to respond to competing health, environment and (sustainable) development impacts and risks. Disease dynamics involves a complex interaction between a host, a pathogen, and their environment, representing one of the largest risks facing the long-term sustainability of agriculture. New airborne inoculum, weather, and satellite-based technology provide new opportunities for combining disease monitoring data and predictive models – but this requires a robust analytical framework. Integrated model-based forecasting frameworks have the potential to improve the timeliness, effectiveness and foresight for controlling crop diseases, while minimizing economic costs and environmental impacts, and yield losses. The feasibility of this approach is investigated involving model and data selection. It is tested against available disease data collected for wheat stripe (yellow) rust (Puccinia striiformis f.sp. tritici) (Pst) fungal disease within southern Alberta, Canada. Two candidate, stochastic models are evaluated; a simpler, site-specific model, and a more complex, spatially-explicit transmission model. The ability of these models to reproduce an observed infection pattern is tested using two climate datasets with different spatial resolution - a reanalysis dataset (~55 km) and weather station network township-aggregated data (~10 km). The complex spatially-explicit model using weather station network data had the highest forecast accuracy. A multi-scale airborne surveillance design that provides data would further improve disease risk forecast accuracy under heterogeneous modeling assumptions. In the future, a model-based forecasting approach, if supported with an airborne surveillance monitoring plan, could be made operational to provide agricultural stakeholders with reliable, cost-effective, and near-real-time information for protecting and sustaining crop production against multiple disease threats.

A modified symbiotic organisms search algorithm for unmanned combat aerial vehicle route planning problem

Route planning is the core component of unmanned combat aerial vehicle (UCAV) systems and the premise for implementation of airborne reconnaissance, surveillance, combat, and other tasks. The purpose is to find the optimal flight route under certain constraints, and its essence is a multi-constrained global optimisation problem. This paper presents a modified symbiotic organisms search algorithm based on the simplex method (SMSOS) to solve the UCAV path planning problem. In addition to the flight environment for the fixed threat area, this paper tested the flight environment of the randomly generated threat area because of the complexity of the actual battlefield threat area. After many simulation tests, it was concluded that SMSOS can find the shortest flight path while avoiding the threat areas. The experimental results show that SMSOS has faster convergence speed, higher precision, and stronger robustness than the other main swarm intelligence algorithms for solving the UCAV flight planning problem.

Model Predictive Control: Simulation Studies for the Implementation on Vertical Take-Off and Landing Lab Prototype

Model Predictive Control (MPC) is one of the advanced control techniques. The ability to act prior to the setpoint change by predicting the future control actions makes MPC a better choice for control applications with predefined setpoint trajectory. This paper is a preliminary work done for the development of control system for an autonomous air-borne surveillance system. A brief comparison among the constrained, unconstrained MPCs and Proportional-Integral-Derivative (PID) controller is presented. Simulation study of the implementation of MPC on the QNET Vertical Take-Off and Landing (VTOL) prototype is investigated in this paper. The software platforms used in this study include MATLAB and Simulink.

Microchannel Plate Imaging Detectors for High Dynamic Range Applications

Microchannel plate (MCP) imaging detectors are widely used in astronomical, biological imaging and remote sensing applications. Photon counting mode imagers with event timing can make use of the high spatial resolution (~10- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$50~\mu \text {m}$ </tex-math></inline-formula> ) and very high time resolution (subnanoseconds) of MCP detectors to enhance the performance of such detectors in dynamic environments (for example airborne surveillance of moving objects, LIDAR, and 3-D topographic imaging). The total information that can be collected is limited by the dynamic range of the detector (cross delay line sensors can support detected photon rates up to ~2 MHz over the entire effective area). The ideal sensor for many applications, such as bright images or fast transient behavior, would combine the desirable attributes of high spatial and time resolution for each detected photon with event rates of 100 MHz or more. A new class of MCPs constructed using atomic layer deposition (ALD) on borosilicate glass microcapillary arrays is providing enhancements towards this goal. The ALD MCP manufacturing process decouples the operational functionalization from the substrate fabrication, opening the door for new resistive layers and high secondary emissive materials. Many improvements over traditional MCPs have been demonstrated, including robust substrates able to withstand high processing temperatures, very low background rates, high stable gains, and low outgassing. Higher global photon counting rates can be supported by ALD MCP detector schemes while greatly improving the instrument lifetime. Additionally, the robustness of these MCPs provide them with surface properties capable of supporting the deposition of a wide range of efficient photocathode materials (III-V), which require high processing temperatures (>500 C), to achieve a high quantum efficiency at near UV to near IR wavelengths. Sealed tube sensors incorporating nano-engineered MCPs can support nonproliferation and remote sensing enabling technologies by advancing the state-of the-art single-photon electro-optical remote sensing detectors. The progress of this effort, including results from the deposition of opaque photocathodes directly onto the front surface of the MCPs and the performance and lifetime characteristics of open faced and seal tube detectors, is reported here.

무인 공중 감시 체계를 위한 이동하는 표적 추적 및 식별 기법

This paper considers vision-based multiple moving target-tracking and target-type recognition methods for unmanned airborne surveillance systems. The detection of moving objects and target-type recognition in a moving image frame are the essential parts of airborne surveillance systems. We propose an optical flow-based object detection method with image stabilization functions to detect moving objects in a moving image frame, and a combination of the Support Vector Machine (SVM) with Convolutional Neural Networks (CNNs) model for the target-type recognition. The experiment of an airborne surveillance scenario using a quadcopter with a camera is conducted to demonstrate the performance of the proposed method.

Integrated Guard Channel Synthesis in AESA based Airborne Surveillance Radar

Integrated Guard Channel Synthesis in AESA based Airborne Surveillance Radar

A novel spatio-temporal saliency approach for robust dim moving target detection from airborne infrared image sequences

Dim moving target detection from infrared image sequences, which lags behind the visual perception ability of humans, has attracted considerable interest from researchers due to its crucial role in airborne surveillance systems. This paper proposes a novel spatio-temporal saliency model to cope with the infrared dim moving target detection problem. Based on a closed-form solution derived from regularized feature reconstruction, a local adaptive contrast operation is proposed, whereby the spatial saliency map and the temporal saliency map can be calculated on the spatial domain and the temporal domain. In order to depict the motion consistency characteristic of the moving target, this paper also proposes a transmission operation to generate the trajectory prediction map. The fused result of the spatial saliency map, the temporal saliency map, and the trajectory prediction map is called the “spatio-temporal saliency map” in this paper, from which the target of interest can be easily segmented. A diverse test dataset comprised of three infrared image sequences under different backgrounds was collected to evaluate the proposed model; and extensive experiments confirmed that the proposed spatio-temporal saliency model can achieve much better detection performance than the state-of-the-art approaches.