Aerial Video Research Articles

災害情報収集におけるドローンの利活用に向けて～空撮映像とWi-Fi情報～

災害情報収集におけるドローンの利活用に向けて～空撮映像とWi-Fi情報～

Multiple vehicle tracking in aerial video sequence using driver behavior analysis and improved deterministic data association

Along with the rapid development of the unmanned aerial vehicle technology, multiple vehicle tracking (MVT) in aerial video sequence has received widespread interest for providing the required traffic information. Due to the camera motion and complex background, MVT in aerial video sequence poses unique challenges. We propose an efficient MVT algorithm via driver behavior-based Kalman filter (DBKF) and an improved deterministic data association (IDDA) method. First, a hierarchical image registration method is put forward to compensate the camera motion. Afterward, to improve the accuracy of the state estimation, we propose the DBKF module by incorporating the driver behavior into the Kalman filter, where artificial potential field is introduced to reflect the driver behavior. Then, to implement the data association, a local optimization method is designed instead of global optimization. By introducing the adaptive operating strategy, the proposed IDDA method can also deal with the situation in which the vehicles suddenly appear or disappear. Finally, comprehensive experiments on the DARPA VIVID data set and KIT AIS data set demonstrate that the proposed algorithm can generate satisfactory and superior results.

Haze removal for unmanned aerial vehicle aerial video based on spatial‐temporal coherence optimisation

Haze removal is a non-trivial work for unmanned aerial vehicle (UAV) aerial video processing, and challenges are mainly attributed to spatial-temporal coherence and computational efficiency. The authors propose a novel dehazing algorithm for hazy UAV aerial video and improve the classical dark channel prior approach with a bright region filling process to alleviate colour distortion in the recovered video, which enhances the spatial consistency. To achieve better temporal coherence, the authors constrain the atmospheric light estimation between adjacent frames by using a temporal filter. The authors also optimise the transmission calculation to reduce computational complexity. Experimental findings show that the proposed algorithm yields results superior to those obtained from previous methods. Compared with that in frame-by-frame dehazing, the processing time in the proposed method is reduced by 74.5%.

改进模型估计的无人机侦察视频快速拼接方法

改进模型估计的无人机侦察视频快速拼接方法

Evaluation of Aerial Video for Unmanned Construction Machine Teleoperation Assistance

Evaluation of Aerial Video for Unmanned Construction Machine Teleoperation Assistance

A thresholding scheme of eliminating false detections on vehicles in wide-area aerial imagery

Post-processings are usually necessary to reduce false detections on vehicles in wide-area aerial imagery. In order to improve the performance of vehicle detection, we propose a two-stage scheme, which consists of a thresholding method by constructing a pixel-weight based thresholding policy to classify pixels in the greyscale feature map of an automatic detection algorithm followed by morphological filtering. We use two aerial videos for performance evaluation, and compare the automatic detection results with the ground-truth objects. We compute average F-score and percentage of wrong classifications towards six detection algorithms before and after applying the proposed scheme. We measure the variation of overlap ratios from detections to objects, and establish sensitivity analysis to evaluate the performance of proposed scheme by combining it on each of two representative algorithms. Simulation results verify both validity and efficiency of the proposed thresholding scheme, also display the difference of detection performance between datasets and among algorithms.

低空视频影像自适应关键帧提取与快速拼接

低空视频影像自适应关键帧提取与快速拼接

Vehicle detection in wide-area aerial imagery: cross-association of detection schemes with post-processings

Post-processing schemes are crucial for object detection algorithms to improve the performance of detection in wide-area aerial imagery. We select appropriate parameters for three algorithms (variational minimax optimisation (Saha and Ray, 2009), feature density estimation (Gleason et al., 2011) and Zheng's scheme by morphological filtering (Zheng et al., 2013)) to achieve the highest average F-score on random sample frames, and then follow the same procedure to implement five post-processing schemes on each algorithm. Two low-resolution aerial videos are used as our datasets to compare automatic detection results with the ground truth objects on each frame. The performance analysis of post-processing schemes on each algorithm are presented under two sets of evaluation metrics.

SAN: an Integrated Unmanned Air Vehicles Interdictor System Concept

The development of unmanned air vehicles (UAVs) has been very rapid recently. The wide accessibility of UAVs has unlocked a great potential for the malicious or accidental damage or destruction of property or other aircraft in flight. UAVs can be used for espionage, contraband, and the trafficking or transport of arms and hazardous materials. Every month sees a growing number of reports of incidents that involve remote-controlled UAVs operated for aerial video recording. These incidents justify undertaking research into minimizing the hazards which UAVs may potentially cause. A concept was developed for a solution dedicated to this problem and comprising an integrated modular anti-UAV system for application in commercial (civilian) markets in the areas of security of mass events, strategic enterprises, and critical strategic infrastructure in Poland. The proposed system is intended to incapacitate any UAVs that breach a predefined air space and bring the incapacitated UAVs safely to the ground. The project is developed jointly by the Institute of Armament Technology at the Military University of Technology (Warsaw, Poland) and Polish company Ellipsis Sp. z o.o.

Artificial intelligence for traffic signal control based solely on video images

ABSTRACTLearning-based traffic control algorithms have recently been explored as an alternative to existing traffic control logics. The reinforcement learning (RL) algorithm is being spotlighted in the field of adaptive traffic signal control. However, no report has described the implementation of an RL-based algorithm in an actual intersection. Most previous RL studies adopted conventional traffic parameters, such as delays and queue lengths to represent a traffic state, which cannot be exactly measured on-site in real time. Furthermore, the traffic parameters cannot fully account for the complexity of an actual traffic state. The present study suggests a novel artificial intelligence that uses only video images of an intersection to represent its traffic state rather than using handcrafted features. In simulation experiments using a real intersection, consecutive aerial video frames fully addressed the traffic state of an independent four-legged intersection, and an image-based RL model outperformed both the actual operation of fixed signals and a fully actuated operation.

Perception of Human Interaction Based on Motion Trajectories: From Aerial Videos to Decontextualized Animations.

People are adept at perceiving interactions from movements of simple shapes, but the underlying mechanism remains unknown. Previous studies have often used object movements defined by experimenters. The present study used aerial videos recorded by drones in a real-life environment to generate decontextualized motion stimuli. Motion trajectories of displayed elements were the only visual input. We measured human judgments of interactiveness between two moving elements and the dynamic change in such judgments over time. A hierarchical model was developed to account for human performance in this task. The model represents interactivity using latent variables and learns the distribution of critical movement features that signal potential interactivity. The model provides a good fit to human judgments and can also be generalized to the original Heider-Simmel animations (1944). The model can also synthesize decontextualized animations with a controlled degree of interactiveness, providing a viable tool for studying animacy and social perception.

Emergency Urban Search

The use of a video camera to support search immediately raises the question “for target location, what is the most effective way of presenting video camera output to a human observer (‘spotter’)”. We examine three presentation modes: (a) unprocessed video output, (b) ‘static visual presentation’ (SVP) in which a series of static views of the search area can be examined in turn while keeping up with drone movement, and (c), a novel mode called ‘Live SVP’ in which the locations sequentially captured by a camera are presented discretely in real time, thereby preserving any movement such as a person waving to attract attention. The dynamics of aerial video were modelled using game development software. The resulting videos were used to support realistic search exercises using human participants. The task attempted by each participant was the identification of lost school children in the simulated environment, using one of the video presentations described above. It was found that the new LSVP viewing mode is superior in those tested for moving targets in a low distraction environment. Another principal finding was that the density of distractors (i.e., non-target objects) had a significant influence on the success of target identification. KEY WORDS: Search, Rescue, Drone, Video, Target, Presentation

Inverse Tsunami Flow Modeling Including Nonequilibrium Sediment Transport, With Application to Deposits From the 2011 Tohoku‐Oki Tsunami

Tsunami deposits provide important clues to understand ancient tsunami events. Several inverse models have been proposed to estimate the magnitude of paleotsunamis from their deposits. However, existing models consider neither nonuniform transport of suspended sediment nor turbulent mixing, which are essential factors governing sedimentation from suspension in tsunami flows. Here we propose a new inverse model of tsunami deposit emplacement, considering both transport of nonuniform suspended load and entrainment of basal sediments. This inversion model requires the spatial distribution of deposit thickness and the pattern of grain size distributions of the tsunami deposit along a 1‐D shoreline‐normal transect as input data. It produces as output run‐up flow velocity, inundation depth, and concentration of suspended sediment. To solve for advection of nonuniform suspended load, a transformed coordinate system is adopted, which increases computational efficiency. Tests of model inversions using artificial data successfully allow reconstruction of the original input values, suggesting the effectiveness of our optimization method. We apply our new inversion model to the 2011 Tohoku‐Oki tsunami deposit on Sendai Plain, Japan. The thickness and grain size distribution of the tsunami deposit was measured along a 4 km long transect normal to the coastline. The result of our inversion fits well with the observations from aerial videos and field surveys. We conclude that this method is suitable for the analysis of ancient tsunami deposits and that it has the advantage of requiring relatively little information about the condition of the emplacing paleotsunami for reconstruction.

Remote Marker-Based Tracking for UAV Landing Using Visible-Light Camera Sensor.

Unmanned aerial vehicles (UAVs), which are commonly known as drones, have proved to be useful not only on the battlefields where manned flight is considered too risky or difficult, but also in everyday life purposes such as surveillance, monitoring, rescue, unmanned cargo, aerial video, and photography. More advanced drones make use of global positioning system (GPS) receivers during the navigation and control loop which allows for smart GPS features of drone navigation. However, there are problems if the drones operate in heterogeneous areas with no GPS signal, so it is important to perform research into the development of UAVs with autonomous navigation and landing guidance using computer vision. In this research, we determined how to safely land a drone in the absence of GPS signals using our remote maker-based tracking algorithm based on the visible light camera sensor. The proposed method uses a unique marker designed as a tracking target during landing procedures. Experimental results show that our method significantly outperforms state-of-the-art object trackers in terms of both accuracy and processing time, and we perform test on an embedded system in various environments.

Moving object detection in top-view aerial videos improved by image stacking

Image stacking is a well-known method that is used to improve the quality of images in video data. A set of consecutive images is aligned by applying image registration and warping. In the resulting image stack, each pixel has redundant information about its intensity value. This redundant information can be used to suppress image noise, resharpen blurry images, or even enhance the spatial image resolution as done in super-resolution. Small moving objects in the videos usually get blurred or distorted by image stacking and thus need to be handled explicitly. We use image stacking in an innovative way: image registration is applied to small moving objects only, and image warping blurs the stationary background that surrounds the moving objects. Our video data are coming from a small fixed-wing unmanned aerial vehicle (UAV) that acquires top-view gray-value images of urban scenes. Moving objects are mainly cars but also other vehicles such as motorcycles. The resulting images, after applying our proposed image stacking approach, are used to improve baseline algorithms for vehicle detection and segmentation. We improve precision and recall by up to 0.011, which corresponds to a reduction of the number of false positive and false negative detections by more than 3 per second. Furthermore, we show how our proposed image stacking approach can be implemented efficiently.

Candidate regions extraction of intruder airplane under complex background for vision‐based sense and avoid system

Candidate regions extraction is a crucial problem in object detection. In this study, a new algorithm is proposed for extracting candidate regions of intruder airplane to assist with vision-based sense and avoid system. A key consideration is that the algorithm works well under complex background. The algorithm contains three parts: roughly determining the area of the intruder airplane on the image, segmenting the detected area into image patches and finally extracting candidate regions from the segmented patches. The cycle-frequency filter is adopted to roughly determine the area of the intruder on the image. The graph-based image segmentation method is used to further separate the determined area into the image patches of the intruder and the image patches of the background. The sliding window technique is employed to obtain the regions of interest (ROIs), and the candidate regions of the intruder airplane are finally extracted by merging the overlapped ROIs. The performance of the algorithm is analysed using a number of aerial video sequences with complex background and different angles of view. The results of the experiments indicate the superiority of the proposed algorithm.

How accurate are small drones for measuring microscopic traffic parameters?

Small Unmanned Aerial Vehicles (sUAV or drones) have been one of the latest tools for monitoring transportation infrastructure and operations. Their lower cost compared to current fixed location camera systems or Manned Aerial Vehicles (MAV) and their ability to read just their view area depending on the situation they face, make them a promising tool of collecting both macroscopic and microscopic data. However, although drone technology and computer vision techniques are advancing fast, there is little information on how accurate and reliable they are for collecting microscopic traffic data. In this paper, we examine the potential of using sUAV as part of the ITS infrastructure as a way of extracting naturalistic trajectory data from aerial video footage from a low volume four-way intersection and a pedestrian passage. Moreover, the accuracy of speed data collected from a drone compared to data collected from an On-Board Diagnostics II (OBD-II) device is examined. For this, a controlled experiment where the vehicle was driven in various speeds and the drone flew in ranging altitudes was conducted. Results show that accuracy is highly dependent on the stabilization of the video and the geo-reference procedure. Moreover, the capabilities of such systems are examined in traffic applications and the way they can be part of future transportation infrastructure is discussed.

A microscopic simulation model for pedestrian-pedestrian and pedestrian-vehicle interactions at crosswalks.

This study aims to develop a microscopic pedestrian behavior model considering various interactions on pedestrian dynamics at crosswalks. Particularly, we take into account the evasion behavior with counter-flow pedestrians, the following behavior with leader pedestrians, and the collision avoidance behavior with vehicles. Aerial video data at one intersection in Beijing, China are extracted for model calibration. A microscopic calibration approach based on maximum likelihood estimation is applied to estimate the parameters of a modified social force model. Finally, we validate step-wise speed, step-wise acceleration, step-wise direction change, crossing time and lane formation phenomenon by comparing the real data and simulation outputs.

Angola Coastal Mapping and Tactics for Oil Spill Response

ABSTRACT The Angolan oil and gas industry workgroup (ACEPA-Associação das Companhias de Exploração e Produção de Angola) continues to collaborate in the development and completion of oil spill response initiatives as responsible operators and committed participants in the IMO-IPIECA Global Initiative for West, Central and Southern Africa (GI WACAF). Between 2012–2015, a comprehensive program of oil spill response planning, geodatabase development, mapping, and guides were completed for the entire coast of Angola. Low altitude aerial video surveys that combine oblique imagery and a verbal commentary record were recorded as the foundation for mapping coastal sensitivities, ESI characterization, shore zone and backshore operational definition, and shoreline segmentation. Other information added to the coastal geodatabase includes operational and logistical considerations such as land access and boat launch locations, proposed sites for staging, holding of temporary wastes, industrial and human use (fishing, recreational), and nearshore access constraints. A key aspect of the shoreline mapping effort was to build the information into a stand-alone pre-shoreline cleanup assessment technique (SCAT)-database that provides layered information for each of the 1999 shoreline segments. Digital video and high-resolution oblique aerial photos for 2400 km of coastline are geo-referenced and integrated into the GIS system with viewer software that allows the user to “fly” the shoreline. Shoreline attributes for biological, socioeconomic, and human use were used to rank 117 sensitive sites. The 45 highest priority sensitive sites were surveyed and detailed geographic response plans (GRPs) for site protection strategies and tactics were compiled into two atlases. The comprehensive coastal characterization, segmentation, and priority site protection plans provide the Angola government and oil industry with spill preparedness tools that are world-class.

Specification and calibration of a microscopic model for pedestrian dynamic simulation at signalized intersections: A hybrid approach

Macroscopic pedestrian models for bidirectional flow analysis encounter limitations in describing microscopic dynamics at crosswalks. Pedestrian behavior at crosswalks is typically characterized by the evasive effect with conflicting pedestrians and vehicles and the following effect with leading pedestrians. This study proposes a hybrid approach (i.e., route search and social force-based approach) for modeling of pedestrian movement at signalized crosswalks. The key influential factors, i.e., leading pedestrians, conflict with opposite pedestrians, collision avoidance with vehicles, and compromise with traffic lights, are considered. Aerial video data collected at one intersection in Beijing, China were recorded and extracted. A new calibration approach based on a genetic algorithm is proposed that enables optimization of the relative error of pedestrian trajectory in two dimensions, i.e., moving distance and angle. Model validation is conducted by comparison with the observed trajectories in five typical cases of pedestrian crossing with or without conflict between pedestrians and vehicles. The characteristics of pedestrian flow, speed, acceleration, pedestrian-vehicle conflict, and the lane formation phenomenon were compared with those from two competitive models, thus demonstrating the advantage of the proposed model.