Proposed Detection Model Research Articles

A hybrid deep learning model for discrimination of physical disturbance and cyber-attack detection in smart grid

A smart grid (SG) consists of an interconnection of an electrical grid, communication, and information networks. The rapid developments of SG technologies have resulted in complex cyber–physical systems. Due to these complexities, the attack surfaces of SGs broaden, and their vulnerabilities to cyber–physical threats increase. SG security systems focus on the protection of significant units and sub-systems of communication and power networks from malicious threats and external attacks. False data injection attack (FDIA) is known as the most severe threat to SG systems. In this paper, a method of optimizing convolutional neural networks — long short-term memory (CNN-LSTM) with particle swarm optimization (PSO) to detect FDIA in the SG system is proposed. This model uses phasor measurement unit (PMU) measurements to detect an abnormal measurement value and determine the type of this anomaly. The complex hyperparameter space of the CNN-LSTM is optimized by the PSO. A detailed numerical comparison is made using the state-of-the-art deep learning (DL) architectures like LSTM, PSO-LSTM, and CNN-LSTM models to verify the accuracy and effectiveness of the proposed model. The results show that the model outperforms other DL models. In addition, the model has a high accuracy rate that provides decision support for the stable and safe operation of SG systems. In this respect, the proposed detection model is a candidate for building a more robust and powerful detection and protection mechanism.

Lightweight and anchor-free frame detection strategy based on improved CenterNet for multiscale ships in SAR images

Ship detection using synthetic aperture radar (SAR) images has important applications in military and civilian fields, but the different sizes of the ship downgrade the detection accuracy of multiscale ships. Aiming at the problem of the poor accuracy and low efficiency of multiscale ship detection in complex scenes, this paper proposes a lightweight and anchor-free frame detection strategy for multiscale ships in SAR images. First, to deal with the problems of limited training samples, different sizes, attitudes, and angles of the ships in SAR images, a data augmentation strategy suitable for SAR images is adopted to expand the training space, followed by multiscale training to enhance the model generalization ability for multiscale ship detection. Second, a lightweight and anchor-free ship detection model based on the improved CenterNet is proposed, which abandons the dense anchor frame generation and extracts the key point of the ships for detection and positioning. Compared with the anchor frame-based detection method, this proposed detection model does not need to use the post-processing method to remove redundant anchor frames, and can accurately locate the center point of the ships with a better detection performance. Third, to reduce the model size and simplify the model parameters, a more lightweight network design is adopted in combination with the characteristics of SAR images. Hence, a residual network (ResNet) with fewer convolutional layers is constructed as the backbone network, and the cross-stage partial network (CSPNet) and spatial pyramid pooling (SPP) network are designed as the bottleneck network. The shallow ResNet can fully extract the SAR image features and reduce the training overfitting, and CSPNet and SPP can effectively combine the low-level image features to obtain the high-level features, reducing the model computation while at the same time enhancing the feature extraction ability. Finally, the evaluation index of the common objects in the context dataset is introduced, which can provide higher-quality evaluation results for ship detection accuracy and provide comprehensive evaluation indicators for multiscale ship detection. Experimental results show that the proposed strategy has the advantages of high detection efficiency, strong detection ability, and good generalization performance, which can achieve real-time and high-precision detection of the multiscale ship in complex SAR images.

Computerized Detection of Calcium Oxalate Crystal Progression

Calcium oxalate crystals in plants can cause health issues such as kidney stones if ingested in large amounts. Calcium oxalate crystallizations affect approximately 4% of plants. Some of these crystallizations are more common, and human and animal ingestion can be avoided if the degree of severity is detected at an early stage. Therefore, in this paper, we present a computerized method for detecting calcium oxalate crystallizations at an early stage, when chances for avoiding it are higher. In our research, electron micrograph processing techniques are used to extract features and measure the degree of crystallization progression in cases of crystalized plants and normal plants. A new fast search algorithm—ODS: One Direction Search—is proposed to detect calcium oxalate crystal progression. The calcium oxalate crystal progression is detected on the basis of electron micrographs of calcium oxalate crystals by means of a temporal test. We employed deep learning for feature extraction. The deep learning technique uses transfer learning, which allows the proposed detection model to be trained on only a small amount of data regarding calcium oxalate crystals for the determination of the presence of calcium oxalate crystals and the severity of the cases. The experimental results, using electron micrographs of 6900 clusters, demonstrated a success rate of 97.5% when detecting cases of calcium oxalate crystals. The simulation results of the new temporal algorithm show an enhancement of the speed by 70% compared to well-known temporal algorithms, and increased accuracy when computing PRSN against other algorithms.

Plant Disease Prediction Using Deep Learning

This study aims to develop an android application to detect and identify plant diseases through deep convolutional neural network. Plant disease is a critical issue in agricultural countries like Bangladesh. Every year production of crops sustains heavy loss due to diseases. It is quite difficult to detect plant diseases with human eyes. So it is essential to build an automated system to detect the diseases. The proposed disease detection model takes an image of a plant leaf as input, processes it and uses deep convolutional neural network to detect and identify the disease. The developed mobile application is user-friendly and can be used by farmers without much technical knowledge.

SPACE: Finding Key-Speaker in Complex Multi-Person Scenes

Intelligent image processing based on deep learning is a recently emerging artificial Intelligence method which is critical in the next generation computing systems. Key-speaker detection using image processing aims to find the speaker who plays the most important role in complex multi-person scenes. However, existing speaker detection methods just judge whether there is somebody talking. To overcome this problem, this paper proposes a novel key-speaker detection approach named Speaker Pose-Attention deteCtion modEl (SPACE). This approach extracts the space information and analyze people's attention gathering to find the key-speaker. The method consists of two main parts. The first part is our proposed Importance Detection Model (IDM) using human pose estimation and an effective evaluation strategy to find candidates of important speaker. The second part consists of Pose-Attention Graph (PAG) and link analysis algorithm. The PAG represents people's attention graph. The link analysis algorithm analyzes the connectivity of this graph to find the key-speaker. Due to the lack of available key-speaker detection datasets, a self-collected dataset containing speech and meeting scenes has been built to estimate SPACE model. The experimental results show that the proposed method can obtain the best detection accuracy compared with other existing methods.

Detection of Random False Data Injection Cyberattacks in Smart Water Systems Using Optimized Deep Neural Networks

A cyberattack detection model based on supervised deep neural network is proposed to identify random false data injection (FDI) on the tank’s level measurements of a water distribution system. The architecture of the neural network, as well as various hyper-parameters, is modified and tuned to acquire the highest detection performance using the smallest size of training data set. The efficacy of the proposed detection model against various activation functions including sigmoid, rectified linear unit, and softmax is examined. Regularization and momentum techniques are applied to update the weights and prohibit overfitting. Moreover, statistical metrics are presented to evaluate the performance and effectiveness of the proposed model in the presence of a range of measurement noise levels. The proposed model is tested for three attack scenarios composed for the battle of the attack detection algorithms. Results confirm that the size of the data sets required to train the neural network (NN) to accomplish the highest levels of accuracy and precision is significantly decreased as the number of hidden layers is increased. The trained 4- and 5-layer deep neural networks are able to detect the readings’ FDIs with 100% precision and accuracy in the presence of 30% background noise in the sensory data.

A cascaded deep-learning-based model for face mask detection

PurposeThis work aims to present a deep learning model for face mask detection in surveillance environments such as automatic teller machines (ATMs), banks, etc. to identify persons wearing face masks. In surveillance environments, complete visibility of the face area is a guideline, and criminals and law offenders commit crimes by hiding their faces behind a face mask. The face mask detector model proposed in this work can be used as a tool and integrated with surveillance cameras in autonomous surveillance environments to identify and catch law offenders and criminals.Design/methodology/approachThe proposed face mask detector is developed by integrating the residual network (ResNet)34 feature extractor on top of three You Only Look Once (YOLO) detection layers along with the usage of the spatial pyramid pooling (SPP) layer to extract a rich and dense feature map. Furthermore, at the training time, data augmentation operations such as Mosaic and MixUp have been applied to the feature extraction network so that it can get trained with images of varying complexities. The proposed detector is trained and tested over a custom face mask detection dataset consisting of 52,635 images. For validation, comparisons have been provided with the performance of YOLO v1, v2, tiny YOLO v1, v2, v3 and v4 and other benchmark work present in the literature by evaluating performance metrics such as precision, recall, F1 score, mean average precision (mAP) for the overall dataset and average precision (AP) for each class of the dataset.FindingsThe proposed face mask detector achieved 4.75–9.75 per cent higher detection accuracy in terms of mAP, 5–31 per cent higher AP for detection of faces with masks and, specifically, 2–30 per cent higher AP for detection of face masks on the face region as compared to the tested baseline variants of YOLO. Furthermore, the usage of the ResNet34 feature extractor and SPP layer in the proposed detection model reduced the training time and the detection time. The proposed face mask detection model can perform detection over an image in 0.45 s, which is 0.2–0.15 s lesser than that for other tested YOLO variants, thus making the proposed detection model perform detections at a higher speed.Research limitations/implicationsThe proposed face mask detector model can be utilized as a tool to detect persons with face masks who are a potential threat to the automatic surveillance environments such as ATMs, banks, airport security checks, etc. The other research implication of the proposed work is that it can be trained and tested for other object detection problems such as cancer detection in images, fish species detection, vehicle detection, etc.Practical implicationsThe proposed face mask detector can be integrated with automatic surveillance systems and used as a tool to detect persons with face masks who are potential threats to ATMs, banks, etc. and in the present times of COVID-19 to detect if the people are following a COVID-appropriate behavior of wearing a face mask or not in the public areas.Originality/valueThe novelty of this work lies in the usage of the ResNet34 feature extractor with YOLO detection layers, which makes the proposed model a compact and powerful convolutional neural-network-based face mask detector model. Furthermore, the SPP layer has been applied to the ResNet34 feature extractor to make it able to extract a rich and dense feature map. The other novelty of the present work is the implementation of Mosaic and MixUp data augmentation in the training network that provided the feature extractor with 3× images of varying complexities and orientations and further aided in achieving higher detection accuracy. The proposed model is novel in terms of extracting rich features, performing augmentation at the training time and achieving high detection accuracy while maintaining the detection speed.

Machine and Deep Learning Techniques for Daytime Fog Detection in Real Time with In-Vehicle Vision Systems Using the SHRP 2 Naturalistic Driving Study Data

The main focus of this study is to develop a system that can accurately detect the presence of fog in real time at a trajectory level. This study leveraged video data from the SHRP 2 Naturalistic Driving Study (NDS). Extensive data reduction steps were taken to classify various levels of foggy weather conditions from the video data to form two unique image data sets. Afterward, features based on the gray level co-occurrence matrix (GLCM) were extracted from the images and used as classification parameters for training support vector machine (SVM) and K-nearest neighbor (K-NN) algorithms. In addition, a convolutional neural network (CNN) was also examined to improve the detection performance. Although the analysis was done initially on a data set consisting of two weather conditions, clear and fog, it has been extended to include different levels of fog, that is, near fog and distant fog. While the accuracy of the first analysis with two categories was approximately 92% and 91% for SVM and K-NN classifiers, respectively, the CNN produced much greater accuracy of 99%. As expected, the accuracy of the second analysis, with more refined weather categories, was relatively lower than the first analysis where CNN, SVM, and K-NN models produced an accuracy of about 98%, 89%, and 88%, respectively. With the rapid advances in connectivity and affordable cameras, the proposed detection models could be integrated into the smartphones of regular road users, creating an effective way to collect real-time road weather information that could be used to improve weather-based variable speed limit (VSL) systems.

An effective detection method for wheat mold based on ultra weak luminescence

It is widely known that mold is one of important indices in assessing the quality of stored wheat. First, mold will decrease the quality of wheat kernels; the wheat kernels infected by mold can produce secondary metabolites, such as aflatoxins, ochratoxin A, zearalenone, fumonisins and so on. Second, the mycotoxins metabolized by mycetes are extremely harmful to humans; once the food or feed is made of by those wheat kernels infected by mold, it will cause serious health problems on human beings as well as animals. Therefore, the effective and accurate detection of wheat mold is vitally important to evaluate the storage and subsequent processing quality of wheat kernels. However, traditional methods for detecting wheat mold mainly rely on biochemical methods, which always involve complex and long pretreatment processes, and waste part of wheat samples for each detection. In view of this, this paper proposes a type of eco-friendly and nondestructive wheat mold detection method based on ultra weak luminescence. The specific implementation process is as follows: firstly, ultra weak luminescence signals of the healthy and the moldy wheat subsamples are measured by a photon analyzer; secondly, the approximate entropy and multiscale approximate entropy are introduced as the main classification features separately; finally, the detection model has been established based on the support vector machine in order to classify two types of wheat subsamples. The receiver operating characteristic curve of the newly established detection model shows that the highest classification accuracy rate can reach 93.1%, which illustrates that our proposed detection model is feasible and promising for detecting wheat mold.

Train Driver Fatigue Detection Using Eye Feature Vector and Support Vector Machine

Fatigue driving is one of the main causes of traffic accidents. The eye features are the important cues of fatigue detection. In order to improve the accuracy and robustness of detection based on a single eye feature, we propose a fatigue detection algorithm based on the eye feature (EFV) vector. Firstly, the coordinates of the eye region were localized with facial landmarks detector and the landmarks geometric relation (LGR) was calculated as a feature value. Secondly, a deep transfer learning network was designed to classify the driver eye state on a small dataset. The probability value of the eyes being open state was calculated. Then an eye feature vector was constructed to overcome the limitations of a single fixed threshold and a support vector machine (SVM) model was trained for eye state classification recognition. Finally, the performance of the proposed detection model was evaluated by the percentage of eyelid closure over time (PERCLOS) criterion. The results show that the accuracy of this model can reach 91.67% on the test database, which is higher than the single-feature-based method. This work lays a foundation for the online fatigue detection of train drivers and the deployment of the train driving monitoring system.

A New Deep Model for Detecting Multiple Moving Targets in Real Traffic Scenarios: Machine Vision-Based Vehicles.

When performing multiple target detection, it is difficult to detect small and occluded targets in complex traffic scenes. To this end, an improved YOLOv4 detection method is proposed in this work. Firstly, the network structure of the original YOLOv4 is adjusted, and the 4× down-sampling feature map of the backbone network is introduced into the neck network of the YOLOv4 model to splice the feature map with 8× down-sampling to form a four-scale detection structure, which enhances the fusion of deep and shallow semantics information of the feature map to improve the detection accuracy of small targets. Then, the convolutional block attention module (CBAM) is added to the model neck network to enhance the learning ability for features in space and on channels. Lastly, the detection rate of the occluded target is improved by using the soft non-maximum suppression (Soft-NMS) algorithm based on the distance intersection over union (DIoU) to avoid deleting the bounding boxes. On the KITTI dataset, experimental evaluation is performed and the analysis results demonstrate that the proposed detection model can effectively improve the multiple target detection accuracy, and the mean average accuracy (mAP) of the improved YOLOv4 model reaches 81.23%, which is 3.18% higher than the original YOLOv4; and the computation speed of the proposed model reaches 47.32 FPS. Compared with existing popular detection models, the proposed model produces higher detection accuracy and computation speed.

Resilient Back Propagation Neural Network Security Model For Containerized Cloud Computing

Cloud-native computing is getting more and more popular in recent years where containerized microservices architectural designs play a central role in building a distributed systems and services. On one hand, they bring convenience and simplicity to build massively scalable distributed cloud-native applications and enable continuous development and delivery for their services. On the other hand, they widen the surface of malicious intrusions, which, in turn, without proper defense mechanisms, lessens their benefits to a certain degree. Among the biggest threats of malicious intrusions are those that belong to the Distributed Denial of Service (DDoS) family. Such type of attacks are challenging because DDoS attacks are elevated hard-to-absorbed threats and have a high degree of variability in types, design, and complexity. In this work, resilient backpropagation neural network was used to build an intelligent network intrusion detection model against the most modernistic DDoS attacks in the cloud-native computing environment. We evaluated our proposed model using the benchmarking Canadian Institute for Cybersecurity evaluation CICDDoS 2019 dataset. Our proposed detection model has achieved high reflective DDoS attack detection. Therefore, it is appropriate to defend against reflective DDoS attacks in containerized cloud-native platforms. Experimental results indicate that the DDoS attack detection accuracy of the proposed resilient neural network model is as high as 97.07% which outperforms most of the well-known learning models mentioned in the most related work. Moreover, the proposed model has achieved a competitive run time performance that highly meets the delay requirements of containerized cloud computing.

A Detection Method for Social Network Images with Spam, Based on Deep Neural Network and Frequency Domain Pre-Processing

As a result of the rapid development of internet technology, images are widely used on various social networks, such as WeChat, Twitter or Facebook. It follows that images with spam can also be freely transmitted on social networks. Most of the traditional methods can only detect spam in the form of links and texts; there are few studies on detecting images with spam. To this end, a novel detection method for identifying social images with spam, based on deep neural network and frequency domain pre-processing, is proposed in this paper. Firstly, we collected several images with embedded spam and combined the DIV2K2017 dataset to build an image dataset for training the proposed detection model. Then, the specific components of the spam in the images were determined through experiments and the pre-processing module was specially designed. Low-frequency domain regions with less spam are discarded through Haar wavelet transform analysis. In addition, a feature extraction module with special convolutional layers was designed, and an appropriate number of modules was selected to maximize the extraction of three different high-frequency feature regions. Finally, the different high-frequency features are spliced along the channel dimension to obtain the final classification result. Our extensive experimental results indicate that the spam element mainly exists in the images as high-frequency information components; they also prove that the proposed model is superior to the state-of-the-art detection models in terms of detection accuracy and detection efficiency.

A Symmetry Evaluation Model for Lower Limb Segments During Upper Limb Loading

Weight-bearing is a common behavior, whereas a load impact on human gait symmetry is seldom considered. This study aims to propose and verify a gait symmetry detection model, by using the gait symmetry under different weight-bearing states to establish a weight-bearing symmetry classification rule, and to measure the consistency of movement through multiple gait symmetry experiments. In details, a classifier combined with statistical learning is presented to quantify the signal difference between left and right legs. The quantization result is incorporated as an index to judge the magnitude of symmetry. Meanwhile, a dynamic time warping (DTW) algorithm is introduced to calculate the signal similarity of two legs in time domain as well as another index. The symmetry trend on leg signals is also considered and described as a third index. By casting these indexes in the proposed detection model, we employ several existing nonlinear classification methods to effectively disclose the difference between legs. Moreover, the information between legs can also be extracted through time warping, yielding the detection model competitive. The validity of the model is verified by hypothetical test. Compared with some state-of-the-art methods, our model achieves an improvement over 5% in the consistency measure of the calf. The symmetry measure of calf and thigh is also test. The proposed method outperforms other methods with the improvement of 12.5%.

Near-surface pedestrian detection method based on deep learning for UAVs in low illumination environments

With the development of unmanned aerial vehicles (UAVs) and computer vision, target detection methods based on UAVs have been increasingly applied in military and civilian fields. Considering the adaptability requirements of low illumination environments such as rain, fog, and night, visible and infrared (IR) sensors are often installed on UAVs to perform in all-weather and all-day conditions. To improve the near-surface detection performance of UAVs in low illumination environments, a pedestrian detection method using image fusion and deep learning is proposed. Visible and IR pedestrian images are collected by the UAV. The corresponding aerial images are registered and annotated. These two different types of aerial images are aligned at the time sequence and matched using the scale invariant feature transform. A U-type generative adversarial network (GAN) is first developed to fuse visible and IR images. A convolutional block attention module is introduced to strengthen the pedestrian target information in the GAN. The spatial domain and channel domain attention mechanisms are proposed to generate color fusion images with rich details and solve the problems of feature extraction as well as fusion rules designed manually in the existing image fusion methods. Then, You Only Look Once Version 3 (YOLOv3)-spatial pyramid pooling combined with transfer learning is adopted using the fused images to train the model on our aerial dataset to verify the pedestrian detection performance. In addition, comparison experiments are carried out. The experimental results demonstrate that the YOLOv3 model is successfully transferred to the target dataset. The performance of the proposed detection model using the fused images for transfer training is the best among the different methods. Finally, the accuracy P, recall R, mean average precision, and F1 score reach 0.804, 0.923, 0.928, and 0.859, respectively.

A Malware Detection Approach Using Autoencoder in Deep Learning

Today, in the field of malware detection, the expanding limitations of traditional detection methods and the increasing accuracy of detection methods designed on the basis of artificial intelligence algorithms are driving research findings in this area in favour of the latter. Therefore, we propose a novel malware detection model in this paper. This model combines a grey-scale image representation of malware with an autoencoder network in a deep learning model, analyses the feasibility of the grey-scale image approach of malware based on the reconstruction error of the autoencoder, and uses the dimensionality reduction features of the autoencoder to achieve the classification of malware from benign software. The proposed detection model achieved an accuracy of 96% and a stable F-score of about 96% by using the Android-side dataset we collected, which outperformed some traditional machine learning detection algorithms.

Meta Structural Learning Algorithm With Interpretable Convolutional Neural Networks for Arrhythmia Detection of Multisession ECG

Detection of arrhythmia of electrocardiogram (ECG) signals recorded within several sessions for each person is a challenging issue, which has not been properly investigated in the past. This arrhythmia detection is challenging since a classification model that is constructed and tested using ECG signals maintains generalization when dealing with unseen samples. This article has proposed a new interpretable meta structural learning algorithm for this challenging detection. Therefore, a compound loss function was suggested including the structural feature extraction fault and space label fault with GUMBEL-SOFTMAX distribution in the convolutional neural network (CNN) models. The collaboration between models was carried out to create learning to learn features in models by transferring the knowledge among them when confronted by unseen samples. One of the deficiencies of a meta-learning algorithm is the non-interpretability of its models. Therefore, to create an interpretability feature for CNN models, they are encoded as the evolutionary trees of the genetic programming (GP) algorithms in this article. These trees learn the process of extracting deep structural features in the course of the evolution in the GP algorithm. The experimental results suggested that the proposed detection model enjoys an accuracy of 98% regarding the classification of 7 types of arrhythmia in the samples of the Chapman ECG dataset recorded from 10646 patients in different sessions. Finally, the comparisons demonstrated the competitive performance of the proposed model concerning the other models based on the big deep models.

Spatiotemporal and kinematic characteristics augmentation using Dual-GAN for ankle instability detection.

Obtaining massive amounts of training data is often crucial for computer-assisted diagnosis using deep learning. Unfortunately, patient data is often small due to varied constraints. We develop a new approach to extract significant features from a small clinical gait analysis dataset to improve computer-assisted diagnosis of Chronic Ankle Instability (CAI) patients. In this paper, we present an approach for augmenting spatiotemporal and kinematic characteristics using the Dual Generative Adversarial Networks (Dual-GAN) to train a series of modified Long Short-Term Memory (LSTM) detection models making the training process more data-efficient. Namely, we use LSTM-, LSTM-Fully Convolutional Networks (FCN)-, and Convolutional LSTM-based detection models to identify the patients with CAI. The Dual-GAN enables the synthesized data to approximate the real data distribution visualized by the t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm. Then we trained the proposed detection models using real data collected from a controlled laboratory study and mixed data from real and synthesized gait features. The detection models were tested in real data to validate the positive role in data augmentation as well as to demonstrate the capability and effectiveness of the modified LSTM algorithm for CAI detection using spatiotemporal and kinematic characteristics in walking. Dual-GAN generated efficient spatiotemporal and kinematic characteristics to augment the training set promoting the performance of CAI detection and the modified LSTM algorithm yielded an enhanced classification outcome to identify those CAI patients from a group of control subjects based on gait analysis data than any previous reports.

One-Stage Arbitrary-Oriented Object Detection With Prediction Decoupling

Arbitrary-oriented objects widely appear in natural scenes, remote sensing images, text images, etc., and thus arbitrary-oriented object detection has received considerable attention. Arbitrary-oriented object detection is a comprehensive task including multi sub-tasks which are boundary regression, classification and orientation prediction. However, since there is feature misalignment among these sub-tasks, the performance of one-stage detectors without fine-tuning process is greatly limited. Thus, in this study, we explore feature misalignment among sub-tasks in one-stage arbitrary-oriented object detection and find that there are relevance and conflicts among the features of category prediction, foreground score prediction, orientation prediction and boundary prediction. In order to alleviate conflicts and extract especial features for each sub-task, we propose a decoupling detection head composed of three branches respectively for object classification, orientation prediction, boundary regression and foreground score prediction. Especially, boundary regression and foreground score prediction share one branch due to their relevance and feature similarity. What’s more, in order to construct an effective detection head for our decoupling model, we design two decoupling detection heads with different network depths. Experiments indicate that appropriate network depth can improve detection performance, and the proposed detection model achieves 68.91% mAP with single-scale testing on the challenging public dataset DOTA-v1.5. Compared with previous competitive method, our model gains 2.05% mAP.

Detection of early bruises on apples using hyperspectral imaging combining with YOLOv3 deep learning algorithm

AbstractThe bruise on apples is a common damage mainly caused during the picking and transportation stages, but its detection robustness has not been satisfactory using the traditional machine learning method, and the precision of bruise detection with short formation time and light degree of bruise is low, and it is easily affected by the variable and complex environment, especially the bright spots. In this paper, a long‐wave near infrared (LW‐NIR 930–2,548 nm) hyperspectral imaging system for acquiring reflectance images of bruised apples were developed in this study. Then the regions of interest (ROIs) of sound and bruised tissues were analyzed and three specific spectral regions were determined. The segmented principal component analysis was performed to select characteristic wavelengths from the three regions. In order to verify the superiority over original hyperspectral data onto deep learning algorithms, the gray‐scale images of characteristic wavelengths (dataset I) and their PC images (dataset II) were input into YOLOv3 network to develop a bruise detection model, respectively. The results showed that all 110 bruise spots of testing set were identified correctly with the YOLOv3 model of dataset I and achieved a F1‐sore of 100% and FPS of 68, while the YOLOv3 model of dataset II and traditional detection method could not eliminate the interference of bright spots and misidentified them as bruises. In short, the YOLOv3 model based on the selected characteristic wavelengths has an excellent potential for on‐line detection of apple bruises.Practical ApplicationsDetection of apple bruises is essential to ensure the fruit quality of the same batch and the income of the farmers. In the past, most bruises were detected using traditional machine learning algorithms combined with hyperspectral imaging technology. This requires a series of cumbersome preprocessing and is susceptible to uneven brightness distribution of reflected image. Compared with the traditional method, our proposed detection model using hyperspectral imaging combining with YOLOv3 deep learning algorithm could obtain better detection accuracy and robustness, and avoid the problem of false detection caused by the interference of bright spots. Simultaneously, the method based on neural network provides a new idea for future multispectral online detection technology.