Real-time Detector Research Articles

Real Time Skin Color Detection Based on Adaptive HSV Thresholding

The detection of human skin color has been studied extensively during the past two decades. It is an essential task for various computer vision applications such as biometric authentication, face/hands tracking and gesture analysis. New machine learning methods are effective for skin color detection. However, they are not suitable for real time applications since they are computationally heavy. A lightweight approach for skin color detection consists of using segmentation rules extracted by an investigation on skin color distribution. The kin appearance varies with diversity of image types, acquisition parameters and scene illumination. There are no general segmentation rules that provide effective skin segmentation for different scene conditions. In this paper we present a real-time skin color detector which adapts itself according to tracked human parts. First, initial thresholds are calculated using two popular skin datasets. Those thresholds can also be calculated quickly using small training sets. The proposed skin color detector showed comparable skin segmentation to DeepLabV3++ application and an improvement in term of F1 measure when compared to methods that relies on static rules.

Fully convolutional line parsing

We present a one-stage Fully Convolutional Line Parsing network (F-Clip) that detects line segments from images. The proposed network is very simple and flexible with variations that gracefully trade off between speed and accuracy for different applications. F-Clip detects line segments in an end-to-end fashion by predicting each line’s center position, length, and angle. We further customize the design of convolution kernels of our fully convolutional network to effectively exploit the statistical priors of the distribution of line angles in real image datasets. We conduct extensive experiments and show that our method achieves a significantly better trade-off between efficiency and accuracy, resulting in a real-time line detector at up to 73 FPS on a single GPU. Such inference speed makes our method readily applicable to real-time tasks without compromising any accuracy of previous methods. Moreover, when equipped with a performance-improving backbone network, F-Clip is able to significantly outperform all state-of-the-art line detectors on accuracy at a similar or even higher frame rate. In other word, under same inference speed, F-Clip always achieving best accuracy compare with other methods. Source code https://github.com/Delay-Xili/F-Clip.

Towards Light-Weight and Real-Time Line Segment Detection

Previous deep learning-based line segment detection (LSD) suffers from the immense model size and high computational cost for line prediction. This constrains them from real-time inference on computationally restricted environments. In this paper, we propose a real-time and light-weight line segment detector for resource-constrained environments named Mobile LSD (M-LSD). We design an extremely efficient LSD architecture by minimizing the backbone network and removing the typical multi-module process for line prediction found in previous methods. To maintain competitive performance with a light-weight network, we present novel training schemes: Segments of Line segment (SoL) augmentation, matching and geometric loss. SoL augmentation splits a line segment into multiple subparts, which are used to provide auxiliary line data during the training process. Moreover, the matching and geometric loss allow a model to capture additional geometric cues. Compared with TP-LSD-Lite, previously the best real-time LSD method, our model (M-LSD-tiny) achieves competitive performance with 2.5% of model size and an increase of 130.5% in inference speed on GPU. Furthermore, our model runs at 56.8 FPS and 48.6 FPS on the latest Android and iPhone mobile devices, respectively. To the best of our knowledge, this is the first real-time deep LSD available on mobile devices.

Visual Object Detection with DETR to Support Video-Diagnosis Using Conference Tools

Real-time multilingual phrase detection from/during online video presentations—to support instant remote diagnostics—requires near real-time visual (textual) object detection and preprocessing for further analysis. Connecting remote specialists and sharing specific ideas is most effective using the native language. The main objective of this paper is to analyze and propose—through DEtection TRansformer (DETR) models, architectures, hyperparameters—recommendation, and specific procedures with simplified methods to achieve reasonable accuracy to support real-time textual object detection for further analysis. The development of real-time video conference translation based on artificial intelligence supported solutions has a relevant impact in the health sector, especially on clinical practice via better video consultation (VC) or remote diagnosis. The importance of this development was augmented by the COVID-19 pandemic. The challenge of this topic is connected to the variety of languages and dialects that the involved specialists speak and that usually needs human translator proxies which can be substituted by AI-enabled technological pipelines. The sensitivity of visual textual element localization is directly connected to complexity, quality, and the variety of collected training data sets. In this research, we investigated the DETR model with several variations. The research highlights the differences of the most prominent real-time object detectors: YOLO4, DETR, and Detectron2, and brings AI-based novelty to collaborative solutions combined with OCR. The performance of the procedures was evaluated through two research phases: a 248/512 (Phase1/Phase2) record train data set, with a 55/110 set of validated data instances for 7/10 application categories and 3/3 object categories, using the same object categories for annotation. The achieved score breaks the expected values in terms of visual text detection scope, giving high detection accuracy of textual data, the mean average precision ranging from 0.4 to 0.65.

Deep Learning for SAR Ship Detection: Past, Present and Future

After the revival of deep learning in computer vision in 2012, SAR ship detection comes into the deep learning era too. The deep learning-based computer vision algorithms can work in an end-to-end pipeline, without the need of designing features manually, and they have amazing performance. As a result, it is also used to detect ships in SAR images. The beginning of this direction is the paper we published in 2017BIGSARDATA, in which the first dataset SSDD was used and shared with peers. Since then, lots of researchers focus their attention on this field. In this paper, we analyze the past, present, and future of the deep learning-based ship detection algorithms in SAR images. In the past section, we analyze the difference between traditional CFAR (constant false alarm rate) based and deep learning-based detectors through theory and experiment. The traditional method is unsupervised while the deep learning is strongly supervised, and their performance varies several times. In the present part, we analyze the 177 published papers about SAR ship detection. We highlight the dataset, algorithm, performance, deep learning framework, country, timeline, etc. After that, we introduce the use of single-stage, two-stage, anchor-free, train from scratch, oriented bounding box, multi-scale, and real-time detectors in detail in the 177 papers. The advantages and disadvantages of speed and accuracy are also analyzed. In the future part, we list the problem and direction of this field. We can find that, in the past five years, the AP50 has boosted from 78.8% in 2017 to 97.8 % in 2022 on SSDD. Additionally, we think that researchers should design algorithms according to the specific characteristics of SAR images. What we should do next is to bridge the gap between SAR ship detection and computer vision by merging the small datasets into a large one and formulating corresponding standards and benchmarks. We expect that this survey of 177 papers can make people better understand these algorithms and stimulate more research in this field.

Graph Neural Networks Based Detection of Stealth False Data Injection Attacks in Smart Grids

False data injection attacks (FDIAs) represent a major class of attacks that aim to break the integrity of measurements by injecting false data into the smart metering devices in power grids. To the best of authors' knowledge, no study has attempted to design a detector that automatically models the underlying graph topology and spatially correlated measurement data of the smart grids to better detect cyber attacks. The contributions of this paper to detect and mitigate FDIAs are twofold. First, we present a generic, localized, and stealth (unobservable) attack generation methodology and publicly accessible datasets for researchers to develop and test their algorithms. Second, we propose a Graph Neural Network (GNN) based, scalable and real-time detector of FDIAs that efficiently combines model-driven and data-driven approaches by incorporating the inherent physical connections of modern AC power grids and exploiting the spatial correlations of the measurement. It is experimentally verified by comparing the proposed GNN based detector with the currently available FDIA detectors in the literature that our algorithm outperforms the best available solutions by 3.14%, 4.25%, and 4.41% in F1 score for standard IEEE testbeds with 14, 118, and 300 buses, respectively.

Electrochemical Characterization of Dopamine in Neural Cells With Flexible Biosensors

Abstract Dopamine is critical for the physiological function and plays a crucial role in the discovery of neurological disorders such as Parkinson's disease. Improving the measurement of this neurotransmitter could improve treatment, diagnosis, and prognosis of neurological disorders. Graphene's outstanding biocompatibility and electrical conductivity have caused it to become a widely used material in cellular interfacing and neurotransmitter characterization. However, graphene has been rarely used to investigate cellular systems after introducing trauma. Sensing dopamine on the cellular level and on the microscale can lead to provide a point-of-care diagnostics for traumatic brain injury patients. The sensitivity of graphene biosensor to different concentrations of dopamine was evaluated in the dynamic range of 0.1–100 µM, and the limit of detection of biosensor was estimated to be 180 µM. In this work, a 3D-printed graphene biosensor was used to characterize the dopamine levels as a real-time detector of neurotransmitters. We used cyclic voltammetry (CV) to measure the response of graphene biosensors to neurotransmitter changes, in addition, to evaluate the effect of UV irradiation as the injury stimulant on the electrical properties of graphene biosensors. We demonstrated that the 3D-printed graphene could detect significant changes in the CV profiles of N27 dopaminergic neural cells cultured on the graphene device in the face of trauma.

License plate detection using YOLO v4

Automatic License Plate Recognition (ALPR) is a sizzling topic in the disciplines of intelligent transportation systems and image recognition. The real-time object detector YOLO (You Only Look Once) - darknet deep learning framework is used in this article to detect car number plates in parking lots in real time. The YOLOv4 deep learning technique was utilized in this proposed strategy to automatically recognize a car's number plate from a video stream. An OCR technique is applied to extract the number from the image of the number plate. The system detects license plates with an accuracy of around 89%.

Automatically search an optimal face detector for a specific deployment environment

Face detection plays an important role in many artificial intelligence applications, such as identity recognition, facial expression recognition, and gender/age recognition. Recently, the development of deep learning techniques has greatly improved face detection’s performance. However, it is still ineffective and time-consuming to manually design hyperparameters of face detectors for different deployment environments with diverse distributions. Besides, due to the limited computation capability, many previous networks are hard to meet the latency requirements in deployment environments, and the improved resolution of current cameras further increases the computation burden. Motivated by the above problems, we propose a searching framework aiming to automatically search a real-time face detector architecture with a fixed complexity constraint, to adapt a specific deployment environment. We model the whole searching space into two parts, including the hyperparameters of the network and the detector. Instead of only searching the network structure, the proposed method considers the whole model’s hyperparameters space which contains the preprocessing and postprocessing parameters. The evolutionary algorithm is employed to find the optimal solution, and new evolutionary operations are proposed to explore architecture space. During the whole searching procedure, we guarantee the computation cost is under the restrictions. The advantages of the proposed framework are that it considers a hard computation cost constraint and the preprocessing and postprocessing hyperparameters, leading to a fully automatic design style and global optimization. Finally, we evaluate the proposed model on the most popular Widerface and FDDB datasets. The proposed detector significantly surpasses the existing lightweight face detectors in the comprehensive performances, and the average latency is twice as shorter as the best competitor.

Hardware-accelerated inference for real-time gravitational-wave astronomy

Computational demands in gravitational-wave astronomy are expected to at least double over the next five years. As kilometre-scale interferometers are brought to design sensitivity, real-time delivery of gravitational-wave alerts will become increasingly important to enable multimessenger follow-up. Here we discuss a novel implementation and deployment of deep learning inference for real-time data denoising and astrophysical source identification. This objective is accomplished using a generic inference-as-a-service model capable of adapting to the future needs of gravitational-wave data analysis. The implementation allows seamless incorporation of hardware accelerators and also enables the use of commercial or private as-a-service computing. Low-latency and offline computing in gravitational-wave astronomy addresses key challenges in scalability and reliability and provides a data analysis platform particularly optimized for deep learning applications. There is a growing need for data cleaning and source identification for gravitational-wave detectors in real time. A deep learning inference-as-a-service framework using off-the-shelf software and hardware can address these challenges in a scalable and reliable way.

Monitoring Occupancy of Bats with Acoustic Data: Power and Sample Size Recommendations

Bats are difficult to study due to their nocturnal, cryptic, and highly vagile nature. Ongoing advances in acoustic recording hardware and call classification software have made species detection and activity monitoring more feasible. Our objectives were to determine the effort necessary to monitor bat assemblages using an occupancy framework and acoustic data and to provide guidelines for researchers interested in developing similar monitoring programs. We collected data at 2 study areas in South Texas from June through September in 2015, 2016, and 2017. We used Pettersson D500X Mk II real-time full-spectrum detectors and classified sound files using SonoBat bat call analysis software. We attempted to collect data during 2 visits to individual sites, with up to 5 consecutive nights per visit each year. We estimated occupancy rates for each species in each study area using occupancy models in Program MARK and included terms to define trends in detection probability through the season. Over the 3 years of our study, we sampled 106 sites with 803 sampling nights and classified a total of 2880 sound files to 7 species. Data sets for 6 of the species supported models indicating that detection probability varied throughout our sampling period. Our results generally indicate that sample sizes between 10 and 20 sites would be required to detect declines in occupancy of 50% over 25 years using 10 nights per site with a starting occupancy rate of 0.70. Detecting declines of 30% in 10 years may require >75 sampling sites. Finally, our analysis shows that recognizing seasonal variation in detection probability, and then timing surveys accordingly, can greatly reduce sample size requirements.

Cyber-secure pneumatic actuator system equipped with encrypted controller and attack detectors

A cyber-secure and flexible actuator is essential for cyber-physical-human systems. The use of wireless communication in such systems improves their usability but adds cybersecurity risks. This study proposes a cyber-secure flexible antagonistic pneumatic artificial muscle (PAM) actuator system. The proposed PAM actuator system has an encrypted proportional-integral controller and a real-time detector for cyberattacks. The controller is concealed in the encrypted control fashion using a homomorphic encryption scheme with security parameter 64 bit, and the control signal is computed over a cipher space. The detector has a threshold to detect significant changes in control inputs caused by cyberattacks such as signal injection and parameter falsification. The experimental results demonstrate that the proposed PAM actuator system achieves the desired control performance and improves the reliability of control operations under cyberattacks. The main contribution of this study is the first implementation of an encrypted controller for a pneumatic actuator to develop a secure PAM actuator with real-time detection of cyberattacks. The results of this study contribute to the future development of safe remote systems that interact with humans in the medical and industrial fields.

Robust fluorescent detection of iodine vapor by a film sensor based on a polymer of intrinsic microporosity

Rapid and high-fidelity detection of radiological iodine species are of extreme importance for the safety of the population and the environment in nuclear fuel reprocessing or in case of nuclear accidents. A simple and universal method to fabricate the sensor to user-friendly devices toward a desired analytical environment would further benefit its applications. Herein, we describe the success of using luminescent spectroscopy for off- and on-line detection of gaseous iodine by a fluorescent film sensor based on a polymer of intrinsic microporosity (PIM-1), a material that possesses intrinsic signal amplification capability and good solubility for processing into desired morphologies. Due to the photoinduced electron transfer (PET) effect, the fluorescence of PIM-1 is quenched drastically upon I2 exposure, but not much altered by H2O, CO2, or H2S. An off-line I2 sensor of free-standing PIM-1 membrane and a real-time flue I2 detector by coating PIM-1 film inside a capillary tube were fabricated and applied. Quick response (few seconds), high signal–noise ratio (five orders of magnitude), low detection limit (few ppb), steam tolerability, as well high reversibility and reusability were achieved, making the material a practically convenient and efficient I2 sensor.

RPEOD: A Real-Time Pose Estimation and Object Detection System for Aerial Robot Target Tracking

Pose estimation and environmental perception are the fundamental capabilities of autonomous robots. In this paper, a novel real-time pose estimation and object detection (RPEOD) strategy for aerial robot target tracking is presented. The aerial robot is equipped with a binocular fisheye camera for pose estimation and a depth camera to capture the spatial position of the tracked target. The RPEOD system uses a sparse optical flow algorithm to track image corner features, and the local bundle adjustment is restricted in a sliding window. Ulteriorly, we proposed YZNet, a lightweight neural inference structure, and took it as the backbone in YOLOV5 (the state-of-the-art real-time object detector). The RPEOD system can dramatically reduce the computational complexity in reprojection error minimization and the neural network inference process; Thus, it can calculate real-time on the onboard computer carried by the aerial robot. The RPEOD system is evaluated using both simulated and real-world experiments, demonstrating clear advantages over state-of-the-art approaches, and is significantly more fast.

Enhancement of photoacoustic spectroscopy with sorption enrichment for ppt-level benzene detection.

A real-time trace gas detector for benzene is demonstrated. The measurement system takes advantage of modest enrichment through short adsorption periods to reach a ppt-level detection limit with a sampling cycle of 90 s, which includes sample adsorption, desorption, and a spectroscopic measurement. Benzene is collected on Tenax TA sorbent for 30 s and then detected from the enriched samples with photoacoustic spectroscopy. High sensitivity is achieved using cantilever-enhanced photoacoustic spectroscopy and a continuous-wave quantum cascade laser emitting at 14.8 µm wavelength, which corresponds to the absorption wavelength of the strongest benzene infrared band. We reach a detection limit of 150 ppt of benzene, over one sampling cycle. Interference from humidity and other common petrochemicals is evaluated.

Why should we care about high temporal resolution monitoring of bioaerosols in ambient air?

This is the first time that atmospheric concentrations of individual pollen types have been recorded by an automatic sampler with 1-hour and sub-hourly resolution (i.e. 1-minute and 1-second data). The data were collected by traditional Hirst type methods and state-of the art Rapid-E real-time bioaerosol detector. Airborne pollen data from 7 taxa, i.e. Acer negundo, Ambrosia, Broussonetia papyrifera, Cupressales (Taxaceae and Cupressaceae families), Platanus, Salix and Ulmus, were collected during the 2019 pollen season in Novi Sad, Serbia. Pollen data with daily, hourly and sub-hourly temporal resolution were analysed in terms of their temporal variability. The impact of turbulence kinetic energy (TKE) on pollen cloud homogeneity was investigated. Variations in Seasonal Pollen Integrals produced by Hirst and Rapid-E show that scaling factors are required to make data comparable. Daily average and hourly measurements recorded by the Rapid-E and Hirst were highly correlated and so examining Rapid-E measurements with sub-hourly resolution is assumed meaningful from the perspective of identification accuracy. Sub-hourly data provided an insight into the heterogenous nature of pollen in the air, with distinct peaks lasting ~5–10 min, and mostly single pollen grains recorded per second. Short term variations in 1-minute pollen concentrations could not be wholly explained by TKE. The new generation of automatic devices has the potential to increase our understanding of the distribution of bioaerosols in the air, provide insights into biological processes such as pollen release and dispersal mechanisms, and have the potential for us to conduct investigations into dose-response relationships and personal exposure to aeroallergens.

Examining YOLO for real-time face-mask detection

COVID-19 pandemic has been affecting human lives since it has been announced by the World Health Organization (WHO). SARS-CoV-2 virus is transferred be means of close contact. Wearing a face mask reduces the infection rate. Several works have been used to employ technology to alleviate its rapid and widespread. Currently, Artificial Intelligence (AI) plays an important role in classifying and detection objects. Several detection algorithms have been recently proposed. In this paper we investigate the performance of the latest versions of You Only Look Once (YOLO) real-time AI-based detector in detecting the wearing of masks.

Expandable Spherical Projection and Feature Concatenation Methods for Real-Time Road Object Detection Using Fisheye Image

Fisheye lens cameras are widely used in such applications where a large field of view (FOV) is necessary. A large FOV can provide an enhanced understanding of the surrounding environment and can be an effective solution for detecting the objects in automotive applications. However, this comes with the cost of strong radial distortions and irregular size of objects depending on the location in an image. Therefore, we propose a new fisheye image warping method called Expandable Spherical Projection to expand the center and boundary regions in which smaller objects are mostly located. The proposed method produces undistorted objects especially in the image boundary and a less unwanted background in the bounding boxes. Additionally, we propose three multi-scale feature concatenation methods and provide the analysis of the influence from the three concatenation methods in a real-time object detector. Multiple fisheye image datasets are employed to demonstrate the effectiveness of the proposed projection and feature concatenation methods. From the experimental results, we find that the proposed Expandable Spherical projection and the LCat feature concatenation yield the best AP performance, which is up to 4.7% improvement compared to the original fisheye image datasets and the baseline model.

SSRFD: single shot real-time face detector

SSRFD: single shot real-time face detector

Adopting the YOLOv4 Architecture for Low-Latency Multispectral Pedestrian Detection in Autonomous Driving.

Detecting pedestrians in autonomous driving is a safety-critical task, and the decision to avoid a a person has to be made with minimal latency. Multispectral approaches that combine RGB and thermal images are researched extensively, as they make it possible to gain robustness under varying illumination and weather conditions. State-of-the-art solutions employing deep neural networks offer high accuracy of pedestrian detection. However, the literature is short of works that evaluate multispectral pedestrian detection with respect to its feasibility in obstacle avoidance scenarios, taking into account the motion of the vehicle. Therefore, we investigated the real-time neural network detector architecture You Only Look Once, the latest version (YOLOv4), and demonstrate that this detector can be adapted to multispectral pedestrian detection. It can achieve accuracy on par with the state-of-the-art while being highly computationally efficient, thereby supporting low-latency decision making. The results achieved on the KAIST dataset were evaluated from the perspective of automotive applications, where low latency and a low number of false negatives are critical parameters. The middle fusion approach to YOLOv4 in its Tiny variant achieved the best accuracy to computational efficiency trade-off among the evaluated architectures.