Mean F1 Score Research Articles

Inference-Optimized High-Performance Photoelectric Target Detection Based on GPU Framework

Deep learning has better detection efficiency than typical methods in photoelectric target detection. However, classical CNNs on GPU frameworks consume too much computing power and memory resources. We propose a multi-stream inference-optimized TensorRT (MSIOT) method to solve this problem effectively. MSIOT uses knowledge distillation to effectively reduce the number of model parameters by layer guidance between CNNs and lightweight networks. Moreover, we use the TensorRT and multi-stream mode to reduce the number of model computations. MSIOT again increases inference speed by 9.3% based on the 4.3–7.2× acceleration of TensorRT. The experimental results show that the model’s mean average accuracy, precision, recall, and F1 score after distillation can reach up to 94.20%, 93.16%, 95.4%, and 94.27%, respectively. It is of great significance for designing a real-time photoelectric target detection system.

Post-revascularization Ejection Fraction Prediction for Patients Undergoing Percutaneous Coronary Intervention Based on Myocardial Perfusion SPECT Imaging Radiomics: a Preliminary Machine Learning Study.

In this study, the ability of radiomics features extracted from myocardial perfusion imaging with SPECT (MPI-SPECT) was investigated for the prediction of ejection fraction (EF) post-percutaneous coronary intervention (PCI) treatment. A total of 52 patients who had undergone pre-PCI MPI-SPECT were enrolled in this study. After normalization of the images, features were extracted from the left ventricle, initially automatically segmented by k-means and active contour methods, and finally edited and approved by an expert radiologist. More than 1700 2D and 3D radiomics features were extracted from each patient's scan. A cross-combination of three feature selections and seven classifier methods was implemented. Three classes of no or dis-improvement (class 1), improved EF from 0 to 5% (class 2), and improved EF over 5% (class 3) were predicted by using tenfold cross-validation. Lastly, the models were evaluated based on accuracy, AUC, sensitivity, specificity, precision, and F-score. Neighborhood component analysis (NCA) selected the most predictive feature signatures, including Gabor, first-order, and NGTDM features. Among the classifiers, the best performance was achieved by the fine KNN classifier, which yielded mean accuracy, AUC, sensitivity, specificity, precision, and F-score of 0.84, 0.83, 0.75, 0.87, 0.78, and 0.76, respectively, in 100 iterations of classification, within the 52 patients with 10-fold cross-validation. The MPI-SPECT-based radiomic features are well suited for predicting post-revascularization EF and therefore provide a helpful approach for deciding on the most appropriate treatment.

DeepVerge: Classification of roadside verge biodiversity and conservation potential

Grasslands are increasingly modified by anthropogenic activities, and species-rich grasslands have become rare habitats in the UK. However, grassy roadside verges often contain conservation priority plant species that should be targeted for protection. Identification of verges with high conservation potential represents a considerable challenge for ecologists, driving the development of automated methods to make up for the shortfall of relevant expertise nationally. Using survey data from 3900 km of roadside verges alongside publicly available street-view imagery, we present DeepVerge: a deep learning-based method that can automatically survey sections of roadside verge based on the presence of positive indicator species. Using images and ground truth survey data from the rural UK county of Lincolnshire, DeepVerge achieved a mean accuracy of 88% and a mean F1 score of 0.82 based on a five-fold cross-validation. We argue that with suitable locale-specific fine-tuning, such a method may be used by local authorities to identify new local wildlife sites, and aid management and environmental planning in line with legal and government policy obligations.

A comparison between Pixel-based deep learning and Object-based image analysis (OBIA) for individual detection of cabbage plants based on UAV Visible-light images

It is challenging to accurately and rapidly extract crops based on the ultra-high spatial resolution images of uncrewed aerial vehicle (UAV). Object-based image analysis (OBIA) was regarded as an effective technique for high-spatial-resolution image classification because of its ability to achieve high accuracy by integrating multi-dimensional features. In recent years, deep learning (DL) techniques, with their ability to automatically learn image features from a large number of images, have shown great potential for crop monitoring. However, a systematic comparison of these two mainstream methods for monitoring the crop phenotype has not been conducted. Therefore, this study compares the performance of two advanced methods, DL and OBIA, in individual cabbage plant detection tasks. The results show that the Mask R-CNN deep learning model outperforms the object-based image analysis-multilevel distance transform watershed segmentation (OBIA-MDTWS) method in crop extraction and counting, with an overall mean F1-Score, accuracy of 2.70, 4.15 percentage points higher, respectively. Moreover, the Mask R-CNN deep learning model has higher computing efficiency, which is 3.74 times higher than the OBIA-MDTWS model. In summary, this study shows that the Mask R-CNN deep learning model performs better in vegetable extraction and quantity estimation, providing technical support for subsequent field nursery management and fine planting.

Optical coherence tomography combined with convolutional neural networks can differentiate between intrahepatic cholangiocarcinoma and liver parenchyma ex vivo

PurposeSurgical resection with complete tumor excision (R0) provides the best chance of long-term survival for patients with intrahepatic cholangiocarcinoma (iCCA). A non-invasive imaging technology, which could provide quick intraoperative assessment of resection margins, as an adjunct to histological examination, is optical coherence tomography (OCT). In this study, we investigated the ability of OCT combined with convolutional neural networks (CNN), to differentiate iCCA from normal liver parenchyma ex vivo.MethodsConsecutive adult patients undergoing elective liver resections for iCCA between June 2020 and April 2021 (n = 11) were included in this study. Areas of interest from resection specimens were scanned ex vivo, before formalin fixation, using a table-top OCT device at 1310 nm wavelength. Scanned areas were marked and histologically examined, providing a diagnosis for each scan. An Xception CNN was trained, validated, and tested in matching OCT scans to their corresponding histological diagnoses, through a 5 × 5 stratified cross-validation process.ResultsTwenty-four three-dimensional scans (corresponding to approx. 85,603 individual) from ten patients were included in the analysis. In 5 × 5 cross-validation, the model achieved a mean F1-score, sensitivity, and specificity of 0.94, 0.94, and 0.93, respectively.ConclusionOptical coherence tomography combined with CNN can differentiate iCCA from liver parenchyma ex vivo. Further studies are necessary to expand on these results and lead to innovative in vivo OCT applications, such as intraoperative or endoscopic scanning.

PCCT: Progressive Class-Center Triplet Loss for Imbalanced Medical Image Classification.

Imbalanced training data in medical image diagnosis is a significant challenge for diagnosing rare diseases. For this purpose, we propose a novel two-stage Progressive Class-Center Triplet (PCCT) framework to overcome the class imbalance issue. In the first stage, PCCT designs a class-balanced triplet loss to coarsely separate distributions of different classes. Triplets are sampled equally for each class at each training iteration, which alleviates the imbalanced data issue and lays solid foundation for the successive stage. In the second stage, PCCT further designs a class-center involved triplet strategy to enable a more compact distribution for each class. The positive and negative samples in each triplet are replaced by their corresponding class centers, which prompts compact class representations and benefits training stability. The idea of class-center involved loss can be extended to the pair-wise ranking loss and the quadruplet loss, which demonstrates the generalization of the proposed framework. Extensive experiments support that the PCCT framework works effectively for medical image classification with imbalanced training images. On four challenging class-imbalanced datasets (two skin datasets Skin7 and Skin 198, one chest X-ray dataset ChestXray-COVID, and one eye dataset Kaggle EyePACs), the proposed approach respectively obtains the mean F1 score 86.20, 65.20, 91.32, and 87.18 over all classes and 81.40, 63.87, 82.62, and 79.09 for rare classes, achieving state-of-the-art performance and outperforming the widely used methods for the class imbalance issue.

Deep learning-based credibility conversation detection approaches from social network.

In recent years, the social networks that have become most exploited sources of information, such as Facebook, Instagram, LinkedIn, and Twitter, have been considered the main sources of non-credible information. False information on these social networks has a negative impact on the credibility of conversations. In this article, we propose a new deep learning-based credibility conversation detection approach in social network environments, called CreCDA. CreCDA is based on: (i) the combination of post and user features in order to detect credible and non-credible conversations; (ii) the integration of multi-dense layers to represent features more deeply and to improve the results; (iii) sentiment calculation based on the aggregation of tweets. In order to study the performance of our approach, we have used the standard PHEME dataset. We compared our approach with the main approaches we have studied in the literature. The results of this evaluation show the effectiveness of sentiment analysis and the combination of text and user levels to analyze conversation credibility. We recorded the mean precision of credible and non-credible conversations at 79%, the mean recall at 79%, the mean F1-score at 79%, the mean accuracy at 81%, and the mean G-Mean at 79%.

Enabling Early Health Care Intervention by Detecting Depression in Users of Web-Based Forums using Language Models: Longitudinal Analysis and Evaluation.

Major depressive disorder is a common mental disorder affecting 5% of adults worldwide. Early contact with health care services is critical for achieving accurate diagnosis and improving patient outcomes. Key symptoms of major depressive disorder (depression hereafter) such as cognitive distortions are observed in verbal communication, which can also manifest in the structure of written language. Thus, the automatic analysis of text outputs may provide opportunities for early intervention in settings where written communication is rich and regular, such as social media and web-based forums. The objective of this study was 2-fold. We sought to gauge the effectiveness of different machine learning approaches to identify users of the mass web-based forum Reddit, who eventually disclose a diagnosis of depression. We then aimed to determine whether the time between a forum post and a depression diagnosis date was a relevant factor in performing this detection. A total of 2 Reddit data sets containing posts belonging to users with and without a history of depression diagnosis were obtained. The intersection of these data sets provided users with an estimated date of depression diagnosis. This derived data set was used as an input for several machine learning classifiers, including transformer-based language models (LMs). Bidirectional Encoder Representations from Transformers (BERT) and MentalBERT transformer-based LMs proved the most effective in distinguishing forum users with a known depression diagnosis from those without. They each obtained a mean F1-score of 0.64 across the experimental setups used for binary classification. The results also suggested that the final 12 to 16 weeks (about 3-4 months) of posts before a depressed user's estimated diagnosis date are the most indicative of their illness, with data before that period not helping the models detect more accurately. Furthermore, in the 4- to 8-week period before the user's estimated diagnosis date, their posts exhibited more negative sentiment than any other 4-week period in their post history. Transformer-based LMs may be used on data from web-based social media forums to identify users at risk for psychiatric conditions such as depression. Language features picked up by these classifiers might predate depression onset by weeks to months, enabling proactive mental health care interventions to support those at risk for this condition.

Round goby [Neogobius melanostomus (Pallas, 1814)], gudgeon (Gobio gobio L.) and bullhead (Cottus gobio L.) show distinct swimming patterns in a vertical slot fish pass

The vertical slots of fish passes represent bottlenecks that must be passed by every fish migrating upstream. The hydraulics in fish passes are well investigated but less is known about the small scale behaviour of fish while passing the vertical slot. Understanding the species-specific swimming behaviour during the passage could allow for creation of future fish passes with hydraulics adapted to the swimming requirements of desired target species. We recorded the swimming trajectories of three fish species as point coordinates per video frame using cameras. Then, two common machine learning algorithms were used to identify species characteristic swimming patterns in the trajectories. A Random Forest model trained on 21 trajectory features revealed that water discharge, the spatial trajectory position, and the trajectory length were most distinct trajectory features among species. The model identified the species with a mean F1 score of 0.86 ± 0.08 SD for round goby [Neogobius melanostomus (Pallas, 1814)], 0.81 ± 0.12 SD for gudgeon (Gobio L.), and 0.58 ± 0.20 SD for bullhead (Cottus gobio L.). A Convolutional Neural Network achieved a mean F1 score of 0.89 ± 0.03 SD for round goby, 0.76 ± 0.05 SD for gudgeon, and 0.67 ± 0.02 SD for bullhead if exclusively trained on the point coordinates of the swimming trajectories. These results demonstrate that fish species exhibit distinct swimming patterns when passing through a vertical slot, and how these patterns can be used for species identification using machine learning algorithms. Because round goby achieved the highest F1 scores, we conclude that round goby showed the most characteristic swimming trajectories among the species tested. Future fish passage research should account for the individual swimming patterns of the fish in these bottleneck flow fields and on adapting the flow to the individual swimming patterns of the target fish. Flow conditions being supportive for swimming patterns of the desired fish could have the potential to improve the river connectivity and thereby support the aquatic biodiversity.

TCNN: A Transformer Convolutional Neural Network for artifact classification in whole slide images

The production of pathological slides is a complex task requiring several physical and chemical procedures that are often done manually. Occasionally, such procedures may end up in patterns (called artifacts) appearing in the images that were not present in the biological tissue but are produced during the slide processing. This paper presents a new method for the automatic detection of such artifacts in pathological images. This detection is seen as a binary classification task that is solved by means of a new Transformer Convolutional Neural Network (TCNN). The proposed method can assist laboratory technicians in avoiding the time-consuming manual labeling of images, preventing the risk of sending artifact patches to pathologists and physicians. The artifact patches include misleading data and should not be considered. In addition, artifact patches jeopardize the accuracy of Computer-Aided Diagnosis (CAD) systems. The proposed method works on the Hue-Saturation-Value colormap layer, which is related to color representation, to improve model robustness against external lighting changes. Spatial Transformer layers are used to make the model spatially invariant on input data, and a set of convolutional layers of EfficientNet Convolutional Neural Network improve pattern recognition. Images used in this research are more than 13,000 and represent 22 types of tissue affected by 7 diverse types of artifacts. Through performance measurement, we evaluate the model using several metrics: Probability Calibration graph, Mean Prediction Probability, Brier Loss, ROC-AUC curve, Confusion Matrix, Accuracy, Sensitivity, Specificity, F1-Score, and Confidence Interval.

Hierarchical strategy for sEMG classification of the hand/wrist gestures and forces of transradial amputees.

The myoelectric control strategy, based on surface electromyographic signals, has long been used for controlling a prosthetic system with multiple degrees of freedom. Several methods classify gestures and force levels but the simultaneous real-time control of hand/wrist gestures and force levels did not yet reach a satisfactory level of effectiveness. In this work, the hierarchical classification approach, already validated on 31 healthy subjects, was adapted for the real-time control of a multi-DoFs prosthetic system on 15 trans-radial amputees. The effectiveness of the hierarchical classification approach was assessed by evaluating both offline and real-time performance using three algorithms: Logistic Regression (LR), Non-linear Logistic Regression (NLR), and Linear Discriminant Analysis (LDA). The results of this study showed the offline performance of amputees was promising and comparable to healthy subjects, with mean F1 scores of over 90% for the "Hand/wrist gestures classifier" and 95% for the force classifiers, implemented with the three algorithms with features extraction (FE). Another significant finding of this study was the feasibility of using the hierarchical classification strategy for real-time applications, due to its ability to provide a response time of 100 ms while maintaining an average online accuracy of above 90%. A possible solution for real-time control of both hand/wrist gestures and force levels is the combined use of the LR algorithm with FE for the "Hand/wrist gestures classifier", and the NLR with FE for the Spherical and Tip force classifiers.

ARTD-Net: Anchor-Free Based Recyclable Trash Detection Net Using Edgeless Module.

Due to the sharp increase in household waste, its separate collection is essential in order to reduce the huge amount of household waste, since it is difficult to recycle trash without separate collection. However, since it is costly and time-consuming to separate trash manually, it is crucial to develop an automatic system for separate collection using deep learning and computer vision. In this paper, we propose two Anchor-free-based Recyclable Trash Detection Networks (ARTD-Net) which can recognize overlapped multiple wastes of different types efficiently by using edgeless modules: ARTD-Net1 and ARTD-Net2. The former is an anchor-free based one-stage deep learning model which consists of three modules: centralized feature extraction, multiscale feature extraction and prediction. The centralized feature extraction module in backbone architecture focuses on extracting features around the center of the input image to improve detection accuracy. The multiscale feature extraction module provides feature maps of different scales through bottom-up and top-down pathways. The prediction module improves classification accuracy of multiple objects based on edge weights adjustments for each instance. The latter is an anchor-free based multi-stage deep learning model which can efficiently finds each of waste regions by additionally exploiting region proposal network and RoIAlign. It sequentially performs classification and regression to improve accuracy. Therefore, ARTD-Net2 is more accurate than ARTD-Net1, while ARTD-Net1 is faster than ARTD-Net2. We shall show that our proposed ARTD-Net1 and ARTD-Net2 methods achieve competitive performance in mean average precision and F1 score compared to other deep learning models. The existing datasets have several problems that do not deal with the important class of wastes produced commonly in the real world, and they also do not consider the complex arrangement of multiple wastes with different types. Moreover, most of the existing datasets have an insufficient number of images with low resolution. We shall present a new recyclables dataset which is composed of a large number of high-resolution waste images with additional essential classes. We shall show that waste detection performance is improved by providing various images with the complex arrangement of overlapped multiple wastes with different types.

Animal Detection and Counting from UAV Images Using Convolutional Neural Networks

In the last decade, small unmanned aerial vehicles (UAVs/drones) have become increasingly popular in the airborne observation of large areas for many purposes, such as the monitoring of agricultural areas, the tracking of wild animals in their natural habitats, and the counting of livestock. Coupled with deep learning, they allow for automatic image processing and recognition. The aim of this work was to detect and count the deer population in northwestern Serbia from such images using deep neural networks, a tedious process that otherwise requires a lot of time and effort. In this paper, we present and compare the performance of several state-of-the-art network architectures, trained on a manually annotated set of images, and use it to predict the presence of objects in the rest of the dataset. We implemented three versions of the You Only Look Once (YOLO) architecture and a Single Shot Multibox Detector (SSD) to detect deer in a dense forest environment and measured their performance based on mean average precision (mAP), precision, recall, and F1 score. Moreover, we also evaluated the models based on their real-time performance. The results showed that the selected models were able to detect deer with a mean average precision of up to 70.45% and a confidence score of up to a 99%. The highest precision was achieved by the fourth version of YOLO with 86%, as well as the highest recall value of 75%. Its compressed version achieved slightly lower results, with 83% mAP in its best case, but it demonstrated four times better real-time performance. The counting function was applied on the best-performing models, providing us with the exact distribution of deer over all images. Yolov4 obtained an error of 8.3% in counting, while Yolov4-tiny mistook 12 deer, which accounted for an error of 7.1%.

A Regional Ionospheric Storm Forecasting Method Using a Deep Learning Algorithm: LSTM

AbstractAn ionospheric storm forecasting method was proposed using a deep learning algorithm, LSTM (long short‐term memory). We used the perturbation index to denote the level of an ionospheric storm, deduced from foF2 data, and helped to remove most of the local time and seasonal variations in the ionosphere. In constructing the model, a number of correlated factors were used as inputs, including the properties of coronal mass ejections, solar flare bursts, interplanetary conditions, and geomagnetic and ionospheric states, and the output was whether an ionospheric storm occurred locally in the next 24 hr. Data sets from 2007 to 2014 were used to train the model, and those from 2015 to 2016 were used for validation. The results showed that the model behaved well in most events. The mean precision rate, recall rate, accuracy, and F1 score of the model were 71.7%, 59.7%, 92.7%, and 65.0% in northern China and 78.9%, 56.3%, 96.3% and 65.0% in southern China, respectively. The LSTM forecasting model performed better than other models such as persistence, multiple‐layer perceptron and support vector machine models. Case studies also showed good performance during geomagnetic storms of different strengths. We believe that this model can be beneficial for functional ionospheric storm operation.

DIAGNOSIS OF COLORECTAL CANCER CASES FROM TOMOGRAPHY IMAGES USING DEEP CONVOLUTIONAL NEURAL NETWORKS

Background/Aim: Tomography imaging is a valuable alternative to colonoscopy for diagnosing colorectal cancer cases especially when colonoscopy is not applicable. Therefore, processing tomography images by computer-aided diagnosis support systems is essential for helping clinicians. The aim of this study is to investigate the applicability of convolutional neural networks (CNN) for diagnosis of colorectal cancer from tomography images. This study uses CNN models to classify abdominal tomography im Methods: ages automatically. The image classication is performed using four benchmark CNN models, namely, MobileNETv2, ResNet101V2, InceptionResnetV2, EfcientNetB2, and a shallower sequential CNN model. The benchmark models are experimented with using transfer learning for two separate cases: with and without data augmentation. According to the results, the proposed sequential CNN model achieve Results: d the highest classication scores with mean accuracy, precision, recall and F1-score of 0.9803, 0.9813, 0.9793, and 0.9803, respectively. It is Conclusion: possible to achieve a better detection performance when a shallower CNN model with lower number of parameters is used. Making prediction with such a model means performing less computation to achieve the result. All in all, computer aided diagnosis of colorectal cancer is possible to help the clinicians during their daily practices

Towards automatic verification of the critical view of the myopectineal orifice with artificial intelligence.

Visualization of key anatomical landmarks is required during surgical Trans Abdominal Pre Peritoneal repair (TAPP)of inguinal hernia. The Critical View of the MyoPectineal Orifice (CVMPO) was proposed to ensure correct dissection. An artificial intelligence (AI) system that automatically validates the presence of key and marks during the procedure is a critical step towards automatic dissection quality assessment and video-based competency evaluation. The aim of this study was to develop an AI system that automatically recognizes the TAPP key CVMPO landmarks in hernia repair videos. Surgical videos of 160TAPP procedures were used in this single-center study. A deep neural network-based object detector was developed to automatically recognize the pubic symphysis, direct hernia orifice, Cooper's ligament, the iliac vein, triangle of Doom, deep inguinal ring, and iliopsoas muscle. The system was trained using 130 videos, annotated and verified by two board-certified surgeons. Performance was evaluated in 30 videos of new patients excluded from the training data. Performance was validated in 2 ways: first, single-image validation where the AI model detected landmarks in a single laparoscopic image (mean average precision (MAP) of 51.2%). The second validation is video evaluation where the model detected landmarks throughout the myopectineal orifice visual inspection phase (mean accuracy and F-score of 77.1 and 75.4% respectively). Annotation objectivity was assessed between 2 surgeons in video evaluation, showing a high agreement of 88.3%. This study establishes the first AI-based automated recognition of critical structures in TAPP surgical videos, and a major step towards automatic CVMPO validation with AI. Strong performance was achieved in the video evaluation. The high inter-rater agreement confirms annotation quality and task objectivity.

Online Learning for Wearable EEG-Based Emotion Classification

Giving emotional intelligence to machines can facilitate the early detection and prediction of mental diseases and symptoms. Electroencephalography (EEG)-based emotion recognition is widely applied because it measures electrical correlates directly from the brain rather than indirect measurement of other physiological responses initiated by the brain. Therefore, we used non-invasive and portable EEG sensors to develop a real-time emotion classification pipeline. The pipeline trains different binary classifiers for Valence and Arousal dimensions from an incoming EEG data stream achieving a 23.9% (Arousal) and 25.8% (Valence) higher F1-Score on the state-of-art AMIGOS dataset than previous work. Afterward, the pipeline was applied to the curated dataset from 15 participants using two consumer-grade EEG devices while watching 16 short emotional videos in a controlled environment. Mean F1-Scores of 87% (Arousal) and 82% (Valence) were achieved for an immediate label setting. Additionally, the pipeline proved to be fast enough to achieve predictions in real-time in a live scenario with delayed labels while continuously being updated. The significant discrepancy from the readily available labels on the classification scores leads to future work to include more data. Thereafter, the pipeline is ready to be used for real-time applications of emotion classification.

Paddy Yield Prediction in Tamilnadu Delta Region Using MLR-LSTM Model

ABSTRACT Crop yield forecasting has been well studied in recent decades and is significant in protecting food security. Crop growth is a complex phenomenon that depends on various factors. Machine learning and deep learning trends have emerged as important innovations in this field. We propose to utilize crop, weather, and soil data from agricultural datasets to evaluate yield prediction behavior. Paddy being a staple food crop in India is chosen for this research. In this paper, we propose hybrid architecture for paddy yield prediction, namely, MLR-LSTM, which combines Multiple Linear Regression and Long Short-Term Memory to utilize their complementary nature. The results are compared with traditional machine learning methods such as Support vector machine, Long short-term memory and Random forest method. Evaluation metrics such as Coefficient of Determination (R2), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Square Error (MSE), F1 score, Recall, and Precision are used to evaluate the hybrid method and traditional models. The results obtained from the proposed hybrid method indicates that the hybrid model delivers better R2, RMSE, MAE, MSE values of 0.93, 0.1549, 0.199, and 0.024 respectively.

Two Novel Models for Traffic Sign Detection Based on YOLOv5s

Object detection and image recognition are some of the most significant and challenging branches in the field of computer vision. The prosperous development of unmanned driving technology has made the detection and recognition of traffic signs crucial. Affected by diverse factors such as light, the presence of small objects, and complicated backgrounds, the results of traditional traffic sign detection technology are not satisfactory. To solve this problem, this paper proposes two novel traffic sign detection models, called YOLOv5-DH and YOLOv5-TDHSA, based on the YOLOv5s model with the following improvements (YOLOv5-DH uses only the second improvement): (1) replacing the last layer of the ‘Conv + Batch Normalization + SiLU’ (CBS) structure in the YOLOv5s backbone with a transformer self-attention module (T in the YOLOv5-TDHSA’s name), and also adding a similar module to the last layer of its neck, so that the image information can be used more comprehensively, (2) replacing the YOLOv5s coupled head with a decoupled head (DH in both models’ names) so as to increase the detection accuracy and speed up the convergence, and (3) adding a small-object detection layer (S in the YOLOv5-TDHSA’s name) and an adaptive anchor (A in the YOLOv5-TDHSA’s name) to the YOLOv5s neck to improve the detection of small objects. Based on experiments conducted on two public datasets, it is demonstrated that both proposed models perform better than the original YOLOv5s model and three other state-of-the-art models (Faster R-CNN, YOLOv4-Tiny, and YOLOv5n) in terms of the mean accuracy (mAP) and F1 score, achieving mAP values of 77.9% and 83.4% and F1 score values of 0.767 and 0.811 on the TT100K dataset, and mAP values of 68.1% and 69.8% and F1 score values of 0.71 and 0.72 on the CCTSDB2021 dataset, respectively, for YOLOv5-DH and YOLOv5-TDHSA. This was achieved, however, at the expense of both proposed models having a bigger size, greater number of parameters, and slower processing speed than YOLOv5s, YOLOv4-Tiny and YOLOv5n, surpassing only Faster R-CNN in this regard. The results also confirmed that the incorporation of the T and SA improvements into YOLOv5s leads to further enhancement, represented by the YOLOv5-TDHSA model, which is superior to the other proposed model, YOLOv5-DH, which avails of only one YOLOv5s improvement (i.e., DH).

Deep learning-based weak micro-defect detection on an optical lens surface with micro vision.

To solve limited efficiency and reliability issues caused by current manual quality control processes in optical lens (OL) production environments, we propose an automatic micro vision-based inspection system named MVIS used to capture the surface defect images and make the OL dataset and predictive inference. Because of low resolution and recognition, OL defects are weak, due to their ambiguous morphology and micro size, making a poor detection effect for the existing method. A deep-learning algorithm for a weak micro-defect detector named ISE-YOLO is proposed, making the best for deep layers, utilizing the ISE attention mechanism module in the neck, and introducing a novel class loss function to extract richer semantics from convolution layers and learning more information. Experimental results on the OL dataset show that ISE-YOLO demonstrates a better performance, with the mean average precision, recall, and F1 score increasing by 3.62%, 6.12% and 3.07% respectively, compared to the YOLOv5. In addition, compared with YOLOv7, which is the latest version of YOLO serials, the mean average precision of ISE-YOLO is improved by 2.58%, the weight size is decreased by more than 30% and the speed is increased by 16%.