Fast-moving Targets Research Articles

Image Quality Assessment in Visual Reinforcement Learning for Fast-moving Targets

Image Quality Assessment in Visual Reinforcement Learning for Fast-moving Targets

Ultrafast near-infrared pyroelectric detector based on inhomogeneous plasmonic metasurface

Pyroelectric (PE) detection technologies have attracted extensive attention due to the cooling-free, bias-free, and broadband properties. However, the PE signals are generated by the continuous energy conversion processes from light, heat, to electricity, normally leading to very slow response speeds. Herein, we design and fabricate a PE detector which shows extremely fast response in near-infrared (NIR) band by combining with the inhomogeneous plasmonic metasurface. The plasmonic effect dramatically accelerates the light-heat conversion process, unprecedentedly improving the NIR response speed by 2−4 orders of magnitude to 22 μs, faster than any reported infrared (IR) PE detector. We also innovatively introduce the concept of time resolution into the field of PE detection, which represents the detector’s ability to distinguish multiple fast-moving targets. Furthermore, the spatially inhomogeneous design overcomes the traditional narrowband constraint of plasmonic systems and thus ensures a wideband response from visible to NIR. This study provides a promising approach to develop next-generation IR PE detectors with ultrafast and broadband responses.

Research on Rapid Recognition of Moving Small Targets by Robotic Arms Based on Attention Mechanisms

For small target objects on fast-moving conveyor belts, traditional vision detection algorithms equipped with conventional robotic arms struggle to capture the long and short-range pixel dependencies crucial for accurate detection. This leads to high miss rates and low precision. In this study, we integrate the traditional EMA (efficient multi-scale attention) algorithm with the c2f (channel-to-pixel) module from the original YOLOv8, alongside a Faster-Net module designed based on partial convolution concepts. This fusion results in the Faster-EMA-Net module, which greatly enhances the ability of the algorithm and robotic technologies to extract pixel dependencies for small targets, and improves perception of dynamic small target objects. Furthermore, by incorporating a small target semantic information enhancement layer into the multiscale feature fusion network, we aim to extract more expressive features for small targets, thereby boosting detection accuracy. We also address issues with training time and subpar performance on small targets in the original YOLOv8 algorithm by improving the loss function. Through experiments, we demonstrate that our attention-based visual detection algorithm effectively enhances accuracy and recall rates for fast-moving small targets, meeting the demands of real industrial scenarios. Our approach to target detection using industrial robotic arms is both practical and cutting-edge.

Quantitative real-time phase microscopy for extended depth-of-field imaging based on the 3D single-shot differential phase contrast (ssDPC) imaging method.

Optical diffraction tomography (ODT) is a promising label-free imaging method capable of quantitatively measuring the three-dimensional (3D) refractive index distribution of transparent samples. In recent years, partially coherent ODT (PC-ODT) has attracted increasing attention due to its system simplicity and absence of laser speckle noise. Quantitative phase imaging (QPI) technologies represented by Fourier ptychographic microscopy (FPM), differential phase contrast (DPC) imaging and intensity diffraction tomography (IDT) need to collect several or hundreds of intensity images, which usually introduce motion artifacts when shooting fast-moving targets, leading to a decrease in image quality. Hence, a quantitative real-time phase microscopy (qRPM) for extended depth of field (DOF) imaging based on 3D single-shot differential phase contrast (ssDPC) imaging method is proposed in this research study. qRPM incorporates a microlens array (MLA) to simultaneously collect spatial information and angular information. In subsequent optical information processing, a deconvolution method is used to obtain intensity stacks under different illumination angles in a raw light field image. Importing the obtained intensity stack into the 3D DPC imaging model is able to finally obtain the 3D refractive index distribution. The captured four-dimensional light field information enables the reconstruction of 3D information in a single snapshot and extending the DOF of qRPM. The imaging capability of the proposed qRPM system is experimental verified on different samples, achieve single-exposure 3D label-free imaging with an extended DOF for 160 µm which is nearly 30 times higher than the traditional microscope system.

Radar Range Deception With Time-Modulated Scatterers

Modern radar systems are designed to have high Doppler tolerance to detect fast-moving targets. This means range and Doppler estimations are inevitably coupled, opening pathways to concealing objects by imprinting artificial Doppler signatures on the reflected echoes. Proper temporal control of the backscattered phase can cause the investigating radar to estimate wrong range and velocity, thus cloaking the real position and trajectory of the scatterer. This deception method is exploited here theoretically for arbitrary Doppler tolerant waveforms and then tested experimentally on an example of the linear frequency modulated radar, which is the most common waveform of that class used in practice. The method allows retaining radio silence with a semi passive (battery assisted) approach that can work well with time-dependent metasurfaces. Furthermore, as an insight into new capabilities, we demonstrate that temporally concealed objects could even be made to appear closer than they truly are without violating the laws of relativity.

An Active Multi-Object Ultrafast Tracking System with CNN-Based Hybrid Object Detection.

This study proposes a visual tracking system that can detect and track multiple fast-moving appearance-varying targets simultaneously with 500 fps image processing. The system comprises a high-speed camera and a pan-tilt galvanometer system, which can rapidly generate large-scale high-definition images of the wide monitored area. We developed a CNN-based hybrid tracking algorithm that can robustly track multiple high-speed moving objects simultaneously. Experimental results demonstrate that our system can track up to three moving objects with velocities lower than 30 m per second simultaneously within an 8-m range. The effectiveness of our system was demonstrated through several experiments conducted on simultaneous zoom shooting of multiple moving objects (persons and bottles) in a natural outdoor scene. Moreover, our system demonstrates high robustness to target loss and crossing situations.

FM-STDNet: High-Speed Detector for Fast-Moving Small Targets Based on Deep First-Order Network Architecture

Identifying objects of interest from digital vision signals is a core task of intelligent systems. However, fast and accurate identification of small moving targets in real-time has become a bottleneck in the field of target detection. In this paper, the problem of real-time detection of the fast-moving printed circuit board (PCB) tiny targets is investigated. This task is very challenging because PCB defects are usually small compared to the whole PCB board, and due to the pursuit of production efficiency, the actual production PCB moving speed is usually very fast, which puts higher requirements on the real-time of intelligent systems. To this end, a new model of FM-STDNet (Fast Moving Small Target Detection Network) is proposed based on the well-known deep learning detector YOLO (You Only Look Once) series model. First, based on the SPPNet (Spatial Pyramid Pooling Networks) network, a new SPPFCSP (Spatial Pyramid Pooling Fast Cross Stage Partial Network) spatial pyramid pooling module is designed to adapt to the extraction of different scale size features of different size input images, which helps retain the high semantic information of smaller features; then, the anchor-free mode is introduced to directly classify the regression prediction information and do the structural reparameterization construction to design a new high-speed prediction head RepHead to further improve the operation speed of the detector. The experimental results show that the proposed detector achieves 99.87% detection accuracy at the fastest speed compared to state-of-the-art depth detectors such as YOLOv3, Faster R-CNN, and TDD-Net in the fast-moving PCB surface defect detection task. The new model of FM-STDNet provides an effective reference for the fast-moving small target detection task.

Remote Sensing Low Signal-to-Noise-Ratio Target Detection Enhancement

In real-time remote sensing application, frames of data are continuously flowing into the processing system. The capability of detecting objects of interest and tracking them as they move is crucial to many critical surveillance and monitoring missions. Detecting small objects using remote sensors is an ongoing, challenging problem. Since object(s) are located far away from the sensor, the target's Signal-to-Noise-Ratio (SNR) is low. The Limit of Detection (LOD) for remote sensors is bounded by what is observable on each image frame. In this paper, we present a new method, a "Multi-frame Moving Object Detection System (MMODS)", to detect small, low SNR objects that are beyond what a human can observe in a single video frame. This is demonstrated by using simulated data where our technology-detected objects are as small as one pixel with a targeted SNR, close to 1:1. We also demonstrate a similar improvement using live data collected with a remote camera. The MMODS technology fills a major technology gap in remote sensing surveillance applications for small target detection. Our method does not require prior knowledge about the environment, pre-labeled targets, or training data to effectively detect and track slow- and fast-moving targets, regardless of the size or the distance.

Estimating Optical Flow with Streaming Perception and Changing Trend Aiming to Complex Scenarios

Most of the recent works on optical flow estimation focus on introducing more effective CNNs or even the Transformer architecture that is quite popular in today’s CV field, but they all still follow the traditional two-frame input structure. In this paper, we argue that optical flow estimation should be more focused on the dynamic information in the image sequence. To that end, we present a straightforward but effective streaming framework equipped with a novel prediction module for capturing optical flow trends, which is a point that has not yet been discussed in previous works. By employing the backbone of the strong baseline to extract the stream of features from three consecutive frames as inputs, this module can predict the current state and update the previous changing trend of optical flow for feature fusion at the next stage. Additionally, we redesign the training loss, providing two new cost functions, in particular, to measure the variations in the optical flow trend. At the forward stage, we deploy a historical buffer to maintain the inference speed so that it is unaffected by additional input frames. Compared with the strong baseline, our method improves the average end-point error by 21.12% on Sintel Clean and 12.94% on Sintel Final. In addition, it also outperforms existing methods in real-world scenarios with low brightness or fast-moving targets.

Model-Rebuilt Disturbance Observer of a Tracking System Based on Acceleration Fusion for Laser Power Transmission

When using laser energy to power long-distance and fast-moving targets, it requires a fast-response and high-precision acquisition, pointing and tracking (APT) system. A fast-steering mirror (FSM) system was used in this paper to track the solar cell array as laser power receiver. The disturbance suppression performance is a key indicator for the FSM stabilization. Generally, a fiber-optic gyroscope (FOG) is employed in the high-sampling-rate velocity loop to enhance the anti-interference ability. However, with the expansion of miniaturized applications, a relatively large, heavy, and high-power FOG is hard to be installed on the small mirror. With this case, this paper used a small-size and high-bandwidth MEMS linear accelerometer in the acceleration loop, substituting the gyroscope. However, the drift and high-frequency noise of the MEMS accelerometer in low frequency will cause APT disturbance. Therefore, an acceleration fusion method with a modified complementary filter was proposed to blend signals of the charge-coupled device and the accelerometers. The fused virtual acceleration can eliminate drift and reduce noise in low frequency and was eventually used in the model-rebuilt disturbance observer loop. At last, the measured results show that the disturbance suppression performance is improved using the presented method, and low-price and small-size MEMS accelerometer can be applied in the APT system.

Through Hawks’ Eyes: Synthetically Reconstructing the Visual Field of a Bird in Flight

Birds of prey rely on vision to execute flight manoeuvres that are key to their survival, such as intercepting fast-moving targets or navigating through clutter. A better understanding of the role played by vision during these manoeuvres is not only relevant within the field of animal behaviour, but could also have applications for autonomous drones. In this paper, we present a novel method that uses computer vision tools to analyse the role of active vision in bird flight, and demonstrate its use to answer behavioural questions. Combining motion capture data from Harris’ hawks with a hybrid 3D model of the environment, we render RGB images, semantic maps, depth information and optic flow outputs that characterise the visual experience of the bird in flight. In contrast with previous approaches, our method allows us to consider different camera models and alternative gaze strategies for the purposes of hypothesis testing, allows us to consider visual input over the complete visual field of the bird, and is not limited by the technical specifications and performance of a head-mounted camera light enough to attach to a bird’s head in flight. We present pilot data from three sample flights: a pursuit flight, in which a hawk intercepts a moving target, and two obstacle avoidance flights. With this approach, we provide a reproducible method that facilitates the collection of large volumes of data across many individuals, opening up new avenues for data-driven models of animal behaviour.

Design of MWIR hyperspectral imagers based on acousto-optic tunable filters

Thermal infrared hyperspectral imagers have significant application values in the field of space optics. However, it is very difficult to obtain the spectral data cube and temperature field information of fast moving targets in a short time. An acousto-optic tunable filter (AOTF) has many advantages, such as fast tuning speed, light weight and no moving parts. It can obtain thermal infrared hyperspectral images of a fast-moving target in a very short integration time. In this paper, a collimating optical system is obtained by splicing optical path a telescope and a secondary imaging mirror. A middle wavelength infrared (MWIR) AOTF is introduced into the collimating optical system to modulate the spectrum and obtain thermal infrared hyperspectral images. The system with a focal length of 18 mm, a relative aperture of 1/2, a field of view (FOV) of 8°, a spectral resolution of 30.08 nm at 4 µm and a 100% cold diaphragm efficiency achieving is taken as a design example. The design results show that the modulation transfer function (MTF) of the system is greater than 0.6 at the Nyquist space frequency of 17 lp/mm, and the imaging quality is close to the diffraction limit. Finally, we develop a prototype of MWIR hyperspectral imager based on AOTFs and do experiments to prove its feasibility.

Adapting the Exploration-Exploitation Balance in Heterogeneous Swarms: Tracking Evasive Targets.

There has been growing interest in the use of multi-robot systems in various tasks and scenarios. The main attractiveness of such systems is their flexibility, robustness, and scalability. An often overlooked yet promising feature is system modularity, which offers the possibility of harnessing agent specialization, while also enabling system-level upgrades. However, altering the agents' capacities can change the exploration-exploitation balance required to maximize the system's performance. Here, we study the effect of a swarm's heterogeneity on its exploration-exploitation balance while tracking multiple fast-moving evasive targets under the cooperative multi-robot observation of multiple moving targets framework. To this end, we use a decentralized search and tracking strategy with adjustable levels of exploration and exploitation. By indirectly tuning the balance, we first confirm the presence of an optimal balance between these two key competing actions. Next, by substituting slower moving agents with faster ones, we show that the system exhibits a performance improvement without any modifications to the original strategy. In addition, owing to the additional amount of exploitation carried out by the faster agents, we demonstrate that a heterogeneous system's performance can be further improved by reducing an agent's level of connectivity, to favor the conduct of exploratory actions. Furthermore, in studying the influence of the density of swarming agents, we show that the addition of faster agents can counterbalance a reduction in the overall number of agents while maintaining the level of tracking performance. Finally, we explore the challenges of using differentiated strategies to take advantage of the heterogeneous nature of the swarm.

A Time-Delay Feedback Neural Network for Discriminating Small, Fast-Moving Targets in Complex Dynamic Environments.

Discriminating small moving objects within complex visual environments is a significant challenge for autonomous micro-robots that are generally limited in computational power. By exploiting their highly evolved visual systems, flying insects can effectively detect mates and track prey during rapid pursuits, even though the small targets equate to only a few pixels in their visual field. The high degree of sensitivity to small target movement is supported by a class of specialized neurons called small target motion detectors (STMDs). Existing STMD-based computational models normally comprise four sequentially arranged neural layers interconnected via feedforward loops to extract information on small target motion from raw visual inputs. However, feedback, another important regulatory circuit for motion perception, has not been investigated in the STMD pathway and its functional roles for small target motion detection are not clear. In this article, we propose an STMD-based neural network with feedback connection (feedback STMD), where the network output is temporally delayed, then fed back to the lower layers to mediate neural responses. We compare the properties of the model with and without the time-delay feedback loop and find that it shows a preference for high-velocity objects. Extensive experiments suggest that the feedback STMD achieves superior detection performance for fast-moving small targets, while significantly suppressing background false positive movements which display lower velocities. The proposed feedback model provides an effective solution in robotic visual systems for detecting fast-moving small targets that are always salient and potentially threatening.

Moving target detection based on background modeling and frame difference

The integration of Unmanned Aerial Vehicle (UAV) technology with moving target detection has diverse applications in military reconnaissance, space remote sensing, and smart cities. Traditional motion-based target detection algorithms offer fast processing speeds but lack accuracy. Deep learning-based algorithms, while accurate for specific targets only, are complex and not suitable for resource-limited UAV platforms and lack real-time performance. Therefore, this study proposes a real-time moving target detection algorithm for UAV platforms based on traditional frame difference algorithm. The purpose of this algorithm is to improve detection accuracy, which has been hindered by the limitations of traditional algorithms caused by camera shake, background changes, and fast-moving targets. The algorithm involves rough background modeling, background updating during subsequent video image sequences, image morphology processing, and background compensation. Experimental results from multiple sets of UAV-borne video data demonstrate the algorithm's high target detection rate, low false alarm rate, and ability to detect moving targets stably in complex environments. The proposed algorithm achieves a speed of 25 FPS and a detection accuracy of 91.8%, meeting real-time and accurate detection requirements.

Resolving Doppler Ambiguity for Fast-moving Targets with FDA-MIMO Radar

Although the problem of range ambiguity is solved with the frequency diverse array (FDA)-multiple-input multiple-output (MIMO) radar by utilizing the degree-of-freedom (DOFs) in the range domain, the phenomenon of Doppler ambiguity occurs for fast-moving target. In this article, an offset frequency alignment (OFA) method is proposed to solve the problem of Doppler ambiguity in FDA-MIMO radar. At the design stage, the Doppler frequency offsets, corresponding to different transmit pulses, are aligned by eliminating the residual phases. In this regard, the point targets can be focused in the transmit spatial frequency domain. Furthermore, the Doppler ambiguity index is estimated after principal velocity compensation resorting to maximum likelihood (ML) criterion, where both the cases of known and unknown range ambiguity indexes are investigated respectively. Moreover, in order to ensure the estimation performance, the frequency offset across the transmit array elements is designed by enlarging the term related to the Doppler frequency offset. At the analysis stage, the performance of Doppler ambiguity index estimation in terms of root mean square error (RMSE) is assessed in comparison with the Cramer-Rao bound (CRB). Numerical results are provided to verify the effectiveness of the proposed method to resolve the Doppler ambiguity.

Dual-pixel tracking of the fast-moving target based on window complementary modulation.

Real-time tracking of fast-moving targets has been utilized in various fields. However, the tracking performance of image-based systems for fast-moving targets is still limited by the huge data throughput and computation. In this study, an image-free target tracking system utilizing a digital micromirror device (DMD) is proposed. The proposed system effectively combines the dual-pixel measurement and window complementary modulation, and the alternating interpolation Kalman filter is implemented to fully use the performance of the DMD and maximize the update rate of the system. The accuracy of the proposed system at the maximum update rate of 22.2 kHz can achieve 0.1 pixels according to the experimental results. Meanwhile, we experimentally demonstrated that the accuracy of the proposed image-free target tracking system is within 0.3 pixels at a maximal velocity of 2 × 104 pixel/s at 22.2 kHz by evaluating the performance of the proposed image-free target tracking system when tracking fast-moving targets with different maximal velocity.

ECO++: Adaptive deep feature fusion target tracking method in complex scene

Efficient Convolution Operator (ECO) algorithms have achieved impressive performances in visual tracking. However, its feature extraction network of ECO is unconducive for capturing the correlation features of occluded and blurred targets between long-range complex scene frames. More so, its fixed weight fusion strategy does not use the complementary properties of deep and shallow features. In this paper, we propose a new target tracking method, namely ECO++, using deep feature adaptive fusion in a complex scene, in the following two aspects: First, we constructed a new temporal convolution mode and used it to replace the underlying convolution layer in Conformer network to obtain an improved Conformer network. Second, we adaptively fuse the deep features, which output through the improved Conformer network, by combining the Peak to Sidelobe Ratio (PSR), frame smoothness scores and adaptive adjustment weight. Extensive experiments on the OTB-2013, OTB-2015, UAV123, and VOT2019 benchmarks demonstrate that the proposed approach outperforms the state-of-the-art algorithms in tracking accuracy and robustness in complex scenes with occluded, blurred, and fast-moving targets.

Localization and tracking of multiple fast moving targets in bistatic MIMO radar

In this paper, we propose a maximum likelihood (ML)-based scheme to accurately localize and track multiple fast-moving targets in multiple-input multiple-output (MIMO) radar systems. For the problem of tracking, the motion of the target will cause a spread spectrum effect, which broadens the beam width and skews the beam steering, rendering accurately localizing and tracking the fast moving targets extremely challenging. To tackle this problem, we use a locally linear model to approximate the real motion states of the target and define a vector to describe its motion state. As such, the motion states of targets at adjacent times can be accurately inferred within a short term based on this approximation model. To incorporate the effect of target motion when performing tracking, we estimate the motion state vector at a reference time from a batch of snapshots by using an ML estimator, instead of estimating the time-varying angular parameters directly as usual. Then the motion state vectors at other times can be inferred based on the estimated one by using the locally linear approximation model. The Cramér-Rao lower bound (CRLB) of the estimated parameters are also derived for performance evaluation. Numerical results validate the performance improvement of the proposed scheme compared to the benchmarks.

Year-long atmospheric characterization effort for laser weapon testing on the Potomac River Test Range at the Naval Surface Warfare Center Dahlgren Division

Characterizing far-field high energy laser (HEL) propagation during laser weapon system (LWS) test events in an open-air environment is challenging due to many variables that affect laser transmission. These influencing variables can be divided into two different categories, the first of which is the test architecture and LWS configuration. Missions to be accomplished by the system under test (SUT) have varying degrees of difficulty, which include fast-moving dynamic targets. The SUT may also have various operating modes, wavelengths, and shifting internal optics. The second category is the underlying physics occurring from inside the laser cavity and along the optical path to the target. Phenomena including—but not limited to—cavity stability, optical physics, optical turbulence, atmospheric absorption and scattering, and thermal blooming will affect laser propagation and the characterization methodology for an LWS. It is important to consider all these factors as weapon systems are tested and scored to requirements and performance specifications for assessing lethality and effectiveness. Therefore, a year-long measurement study was conducted on the Potomac River Test Range (PRTR) to assess the optical turbulence conditions of the laser range in support of LWS test and evaluation (T&E) events. Experimental data is analyzed for averaged optical turbulence seasonal trends on a complex nonuniform range. The results are used to coordinate optimal test event timing, predict a laser’s far-field properties at the laser termination site, and generate atmospheric forecasts in conjunction with HEL and atmospheric modeling software. The results will contribute to improving future HEL testing methodology and examining accuracy of models producing optical turbulence calculations.