Deep Learning-based Recognition Research Articles

Exploiting Optimal Self-Attention Deep Learning-based Recognition of Textual Emotions for Disabled Persons

Exploiting Optimal Self-Attention Deep Learning-based Recognition of Textual Emotions for Disabled Persons

Learning-assisted specklegram analysis for recognition of simultaneous weights on multimode optical fiber

A deep learning-based recognition of multimode fiber (MMF) specklegrams for various simultaneous weights is presented in this work. Five different random locations have been considered along the length of MMF and the specklegram images are recorded corresponding to seven different combinations of random simultaneous weights applied at these locations. A popular deep learning convolutional neural network (CNN) model, VGG-16 is employed on these images for the recognition of these seven weight combinations. The impact of acoustic vibrations, laser power, external temperature, and image sizes on the recognition accuracy is examined. A 100% recognition accuracy is attained and a negligible accuracy variation of ∼1.9% for acoustic vibrations as well as for changing laser power is observed, whereas a drastic fall in accuracy is observed in case of change in image sizes less than 80 × 80 pixels. Also, a negligible variation of ∼2% is observed for the applied external temperature. The heart of our work lies in the accumulation of a diverse, large volume of specklegram dataset by virtue of conducting brute force experiments that take care of eradication of model overfitting. The proposed proof-of-concept scheme might be useful for low-cost, efficacious, self-assisted multi-weight analysis in structural health monitoring.

Deep migration learning-based recognition of diseases and insect pests in Yunnan tea under complex environments

BackgroundThe occurrence, development, and outbreak of tea diseases and pests pose a significant challenge to the quality and yield of tea, necessitating prompt identification and control measures. Given the vast array of tea diseases and pests, coupled with the intricacies of the tea planting environment, accurate and rapid diagnosis remains elusive. In addressing this issue, the present study investigates the utilization of transfer learning convolution neural networks for the identification of tea diseases and pests. Our objective is to facilitate the accurate and expeditious detection of diseases and pests affecting the Yunnan Big leaf kind of tea within its complex ecological niche.ResultsInitially, we gathered 1878 image data encompassing 10 prevalent types of tea diseases and pests from complex environments within tea plantations, compiling a comprehensive dataset. Additionally, we employed data augmentation techniques to enrich the sample diversity. Leveraging the ImageNet pre-trained model, we conducted a comprehensive evaluation and identified the Xception architecture as the most effective model. Notably, the integration of an attention mechanism within the Xeption model did not yield improvements in recognition performance. Subsequently, through transfer learning and the freezing core strategy, we achieved a test accuracy rate of 98.58% and a verification accuracy rate of 98.2310%.ConclusionsThese outcomes signify a significant stride towards accurate and timely detection, holding promise for enhancing the sustainability and productivity of Yunnan tea. Our findings provide a theoretical foundation and technical guidance for the development of online detection technologies for tea diseases and pests in Yunnan.

PCINet: a Prototype- and Concept-based Interpretable Network for Mutli-scene Recognition

Abstract. With the development of remote sensing techniques, a large number of high-resolution aerial images is now available and benefit many applications. Multi-scene recognition plays a key role in applying remote sensing images to these applications, which refers to predicting multiple scenes coexisted in an aerial image and has attracted an increasing attention. Recently, most researchers tend to invent deep learning-based recognition models and has gained great achievements. However, few efforts have been deployed to explaining the success of deep neural networks in multi-scene recognition. To address this, we introduce concept bottleneck model (CBM) to interpreting model performance and propose a novel network, namely Prototype- and Concept-based Interpretable Network (PCINet), that projects aerial imagery into a prototype-concept memory bank and encode their correlations for explaining how a network can identify coexisting scenes in an aerial image. Specifically, the proposed network mainly consists of two branches: prototype matching that measures similarity scores between image features and scene prototypes, and concept bottleneck branches that aligned image features to textual embeddings and compute their relations with concept embeddings. Afterwards, Outputs are integrated for inferring scene categories. Experimental results show that the model enhances interpretability, providing valuable insights for urban planning and resource management, thereby bridging the gap between deep learning models and practical applications.

Deep Learning-Based Recognition of Plant Leaf Diseases

Early and accurate disease detection is crucial for effective crop management. This paper proposes a deep learning-based system for automatic plant disease detection using image processing techniques. Our approach leverages Convolutional Neural Networks (CNNs) to analyze images of leaves. The CNN is trained to identify various diseases based on visual symptoms like black and brown spots, mimicking the methods employed by human professionals. This system offers real-time crop monitoring and eliminates the need for farmers to switch between multiple disease control strategies without proper diagnosis. By analyzing image features, the system automatically categorizes the disease present in a captured image. Our research demonstrates that this approach achieves high accuracy in disease detection, potentially leading to improved crop health and yield. Keywords— Deep learning, Convolutional Neutral Network (CNN),Plant Leaf Diseases Image.

Deep learning-based recognition system for pashto handwritten text: benchmark on PHTI.

This article introduces a recognition system for handwritten text in the Pashto language, representing the first attempt to establish a baseline system using the Pashto Handwritten Text Imagebase (PHTI) dataset. Initially, the PHTI dataset underwent pre-processed to eliminate unwanted characters, subsequently, the dataset was divided into training 70%, validation 15%, and test sets 15%. The proposed recognition system is based on multi-dimensional long short-term memory (MD-LSTM) networks. A comprehensive empirical analysis was conducted to determine the optimal parameters for the proposed MD-LSTM architecture; Counter experiments were used to evaluate the performance of the proposed system comparing with the state-of-the-art models on the PHTI dataset. The novelty of our proposed model, compared to other state of the art models, lies in its hidden layer size (i.e., 10, 20, 80) and its Tanh layer size (i.e., 20, 40). The system achieves a Character Error Rate (CER) of 20.77% as a baseline on the test set. The top 20 confusions are reported to check the performance and limitations of the proposed model. The results highlight complications and future perspective of the Pashto language towards the digital transition.

Deep Learning-Based Recognition and Visualization of Human Motion Behavior

Human behavior recognition refers to the classification task of identifying the specific actions of human characters based on the characteristics of human body and the completed actions through a specific algorithm. It has a wide range of applications in intelligent surveillance, video retrieval and so on. The main challenge in this direction is to accurately extract the semantic information of each behavior to describe its dynamic changes in space and time. Therefore, this article introduces the latest research progress in the field of human behavior recognition. Through deep learning techniques, particularly convolutional neural networks and recurrent neural networks, human movements in video data can be effectively identified. However, deep learning models lack interpretability, which can be a challenge in practical applications. The researchers also introduce the application of traditional methods and deep learning-based methods to human behavior recognition, and explore the advantages of deep learning models in processing multi-time scale information and introducing attention mechanisms. Finally, the paper summarizes the potential of deep learning technology combined with multimodal data in behavioral analysis, and provides prospects for applications in smart fitness, health care and other fields.

Domain generalization across tumor types, laboratories, and species — Insights from the 2022 edition of the Mitosis Domain Generalization Challenge

Recognition of mitotic figures in histologic tumor specimens is highly relevant to patient outcome assessment. This task is challenging for algorithms and human experts alike, with deterioration of algorithmic performance under shifts in image representations. Considerable covariate shifts occur when assessment is performed on different tumor types, images are acquired using different digitization devices, or specimens are produced in different laboratories. This observation motivated the inception of the 2022 challenge on MItosis Domain Generalization (MIDOG 2022). The challenge provided annotated histologic tumor images from six different domains and evaluated the algorithmic approaches for mitotic figure detection provided by nine challenge participants on ten independent domains. Ground truth for mitotic figure detection was established in two ways: a three-expert majority vote and an independent, immunohistochemistry-assisted set of labels. This work represents an overview of the challenge tasks, the algorithmic strategies employed by the participants, and potential factors contributing to their success. With an F1 score of 0.764 for the top-performing team, we summarize that domain generalization across various tumor domains is possible with today’s deep learning-based recognition pipelines. However, we also found that domain characteristics not present in the training set (feline as new species, spindle cell shape as new morphology and a new scanner) led to small but significant decreases in performance. When assessed against the immunohistochemistry-assisted reference standard, all methods resulted in reduced recall scores, with only minor changes in the order of participants in the ranking.

Research on the Recognition Method of Dial Scales for Arrester Pointer Instruments Based on Deep Learning

To address the recognition challenges faced by arrester pointer instruments’ dial scales in various scenarios, this paper introduces a deep learning-based recognition method for pointer instrument scales. An attention module is integrated into the YOLOv5 network architecture, enhancing the accuracy and robustness of the model. After correcting the dial, dial recognition is conducted with OpenCV to achieve precise identification of the instrument scales. The proposed method was tested using images of arrester pointer instruments against diverse backgrounds. The experimental results demonstrate that the method processes instrument data images in an average time of 0.662 s and achieves a successful recognition rate of 96% with an average error of 0.923%. This method provides a rapid and efficient approach for recognizing instrument scales and offers a novel solution for identifying similar types of instruments.

DREAM: Deep Learning-Based Recognition of Emotions From Multiple Affective Modalities Using Consumer-Grade Body Sensors and Video Cameras

DREAM: Deep Learning-Based Recognition of Emotions From Multiple Affective Modalities Using Consumer-Grade Body Sensors and Video Cameras

Decoding the Unspoken: Deep Learning-Based Recognition of Silently Spoken English Vowels Using sEMG Signals

Decoding the Unspoken: Deep Learning-Based Recognition of Silently Spoken English Vowels Using sEMG Signals

A Forest Fire Recognition Method Based on Modified Deep CNN Model

Controlling and extinguishing spreading forest fires is a challenging task that often leads to irreversible losses. Moreover, large-scale forest fires generate smoke and dust, causing environmental pollution and posing potential threats to human life. In this study, we introduce a modified deep convolutional neural network model (MDCNN) designed for the recognition and localization of fire in video imagery, employing a deep learning-based recognition approach. We apply transfer learning to refine the model and adapt it for the specific task of fire image recognition. To combat the issue of imprecise detection of flame characteristics, which are prone to misidentification, we integrate a deep CNN with an original feature fusion algorithm. We compile a diverse set of fire and non-fire scenarios to construct a training dataset of flame images, which is then employed to calibrate the model for enhanced flame detection accuracy. The proposed MDCNN model demonstrates a low false alarm rate of 0.563%, a false positive rate of 12.7%, a false negative rate of 5.3%, and a recall rate of 95.4%, and achieves an overall accuracy of 95.8%. The experimental results demonstrate that this method significantly improves the accuracy of flame recognition. The achieved recognition results indicate the model’s strong generalization ability.

Marine vessel detection dataset and benchmark for unmanned surface vehicles

Deep learning-based visual perception of an unmanned surface vehicle (USV) essentially benefits from informative images. It is unfortunately short of available datasets of marine vessel targets for visual perception system of USVs. In this paper, a marine vessel detection dataset, termed MVDD13, is exclusively established by deploying 35,474 images which are accurately annotated to be 13-category vessels. The proposed MVDD13 is equipped with benchmark features by sufficiently taking into account the reality of category proportion, image diversity, sample independence, and background confusion, etc., thereby facilitating rather deep information for training and testing a robust detector. To evaluate benchmark quality, the state-of-the-art deep learning detectors conducting on the MVDD13 show that individual advantages can be consistently reflected. Furthermore, by cross-dataset training/testing, intensive comparisons on general COCO and specific SeaShips datasets reveal that the MVDD13 can significantly fertilize deep learning-based recognition performance in terms of mAP@.5, mAP@.75 and mAP@[.5:.95]. All images and annotations can be downloaded via https://github.com/yyuanwang1010/MVDD13.

Optimal Deep Learning-Based Recognition Model for EEG Enabled Brain-Computer Interfaces Using Motor-Imagery

Abstract Brain-Computer Interfaces (BCIs) facilitate the translation of brain activity into actionable commands and act as a crucial link between the human brain and the external environment. Electroencephalography (EEG)-based BCIs, which focus on motor imagery, have emerged as an important area of study in this domain. They are used in neurorehabilitation, neuroprosthetics, and gaming, among other applications. Optimal Deep Learning-Based Recognition for EEG Signal Motor Imagery (ODLR-EEGSM) is a novel approach presented in this article that aims to improve the recognition of motor imagery from EEG signals. The proposed method includes several crucial stages to improve the precision and effectiveness of EEG-based motor imagery recognition. The pre-processing phase starts with the Variation Mode Decomposition (VMD) technique, which is used to improve EEG signals. The EEG signals are decomposed into different oscillatory modes by VMD, laying the groundwork for subsequent feature extraction. Feature extraction is a crucial component of the ODLR-EEGSM method. In this study, we use Stacked Sparse Auto Encoder (SSAE) models to identify significant patterns in the pre-processed EEG data. Our approach is based on the classification model using Deep Wavelet Neural Network (DWNN) optimized with Chaotic Dragonfly Algorithm (CDFA). CDFA optimizes the weight and bias values of the DWNN, significantly improving the classification accuracy of motor imagery. To evaluate the efficacy of the ODLR-EEGSM method, we use benchmark datasets to perform rigorous performance validation. The results show that our approach outperforms current methods in the classification of EEG motor imagery, confirming its promising performance. This study has the potential to make brain-computer interface applications in various fields more accurate and efficient, and pave the way for brain-controlled interactions with external systems and devices.

Highly elastic, frost-resistant and antimicrobial flexible sensor inspired by ramen noodles for human motion detection and deep learning of handwritten content

Biomaterial-based hydrogels have gained significant attention in the development of flexible multifunctional electronic devices for the future. However, it is a great challenge to prepare biomaterial-based hydrogels with high toughness, electrical conductivity and frost-resistant by a simple and low-cost way. Meanwhile, starch bio-based hydrogels have emerged as a popular area of research due to their abundant sources and degradability. Unfortunately, these hydrogels often exhibit poor mechanical properties and freeze resistance, which restrict their practical applications. Taking inspiration from the traditional food ramen, where a simple mixture of noodles can become remarkably stretchable and resilient, our study aims to enhance the performance of hydrogels by leveraging their toughening mechanism. We achieve this by combining polyvinyl alcohol (PVA), branched chain starch (Amy), wheat protein (WP), ethylene glycol (EG) with saline. The Amy/PVA/EG/WP/S (APEWS) organohydrogels demonstrated excellent mechanical properties (1.12 MPa) due to the combination of PVA as a hard backbone, a rich protein network, diverse functional groups, and saline components. Additionally, the binary solvent consisting of water and EG allowed the hydrogel to function effectively even at low temperatures (-20 °C). The APEWS organohydrogel exhibited promising sensing capability (GF = 1.46) and sensing stability (>500 times) as a human wearable strain sensor, making it suitable for detecting human motion and the hydrogel showed excellent antimicrobial properties against Escherichia coli (Gram-negative bacteria). Furthermore, we employed a Long Short-Term Memory (LSTM) artificial neural network for deep learning-based recognition of written content. The network achieved an accuracy of over 95.5% for different English letters and 100% for symbols. These results highlight the potential of this technology in applications such as writing sensing and information encryption. In conclusion, with the simple preparation method and low cost, APEWS organohydrogels are expected to be the next generation of green and sustainable bio-based strain sensors.

Detection of Pig Movement and Aggression Using Deep Learning Approaches.

Motion and aggressive behaviors in pigs provide important information for the study of social hierarchies in pigs and can be used as a selection indicator for pig health and aggression parameters. However, relying only on visual observation or surveillance video to record the number of aggressive acts is time-consuming, labor-intensive, and lasts for only a short period of time. Manual observation is too short compared to the growth cycle of pigs, and complete recording is impractical in large farms. In addition, due to the complex process of assessing the intensity of pig aggression, manual recording is highly influenced by human subjective vision. In order to efficiently record pig motion and aggressive behaviors as parameters for breeding selection and behavioral studies, the videos and pictures were collected from typical commercial farms, with each unit including 8~20 pigs in 7~25 m2 space; they were bred in stable social groups and a video was set up to record the whole day's activities. We proposed a deep learning-based recognition method for detecting and recognizing the movement and aggressive behaviors of pigs by recording and annotating head-to-head tapping, head-to-body tapping, neck biting, body biting, and ear biting during fighting. The method uses an improved EMA-YOLOv8 model and a target tracking algorithm to assign a unique digital identity code to each pig, while efficiently recognizing and recording pig motion and aggressive behaviors and tracking them, thus providing statistics on the speed and duration of pig motion. On the test dataset, the average precision of the model was 96.4%, indicating that the model has high accuracy in detecting a pig's identity and its fighting behaviors. The model detection results were highly correlated with the manual recording results (R2 of 0.9804 and 0.9856, respectively), indicating that the method has high accuracy and effectiveness. In summary, the method realized the detection and identification of motion duration and aggressive behavior of pigs under natural conditions, and provided reliable data and technical support for the study of the social hierarchy of pigs and the selection of pig health and aggression phenotypes.

A Hybrid Intelligence (HI) System for Segmenting Rectoprostatic Spacer Gel and Key OARs on CT Images for Prostate Cancer Radiation Therapy Planning

Our hybrid intelligence (HI) system, combining natural and artificial intelligence, is effective for auto-contouring H&N and thorax organs at risk (OARs) for radiation therapy (RT) planning with FDA 510(k) clearance. The purpose of this study is to test the HI system to segment a commercially available retroprostatic hyaluronic acid spacer gel (RSG) and pelvic OARs in planning CT images for prostate cancer RT. HI can achieve clinically acceptable auto segmentation for tissue-equivalent RSG in this domain. RSG is injected in the peri-rectal space in men with prostate cancer prior to RT to minimize rectal toxicity. 190 patients with prostate cancer were included in this post-hoc image analysis from a multi-center, prospective, randomized trial, with 136 in the spacer arm. The HI system has 3 steps: rough recognition from fuzzy model (FM) based automatic anatomy recognition (AAR-R), deep learning-based recognition (DL-R) refinement, and deep learning-based delineation (DL-D) to contour objects guided by the recognition results. FM encodes high level 3D anatomy knowledge of object shape and its relationship with other OARs; DL-R and DL-D focus on pixel-level details. The 190 studies are divided into disjoint training (100) and testing (90) subsets. 100 samples are used in DL-R and DL-D training, with 45 to build the FM for AAR-R. RSG and 4 other OARs (pelvic skin, prostate, bladder, rectum) are contoured. Location error (LE) is used to evaluate recognition; Dice coefficient (DC) and Hausdorff distance (HD) are employed to evaluate delineation. Acceptability scores (AS) (range 1-5, 1 for poor quality, 5 for best quality) from an observer study are recorded for HI-output and ground truth masks of RSG for assessing segmentation quality. The HI system achieves highest DC (0.94±0.07) and lowest HD (1.96±1.61 mm) for bladder, for rectum and prostate similar DC (0.82±0.08) and HD (2.62±1.65mm), for RSG, the most challenging object, a good DC close to 0.7 (0.67±0.10) and excellent HD (2.66±1.44mm). AS for auto-segmentations (3.86±0.85) were significantly better than those for ground truth segmentations (3.45±1.00) (p = 0.02, paired t-test). Table 1 summarizes results. The HI system achieves clinically acceptable segmentations for pelvic OARs and significantly better acceptability of segmentation of RSG compared to clinically performed ground truth segmentations. This has implications in improving efficiency and accuracy of CT-based RT planning in patients with prostate cancer.

Deep Learning-Based Recognition of Periodontitis and Dental Caries in Dental X-ray Images.

Dental X-ray images are important and useful for dentists to diagnose dental diseases. Utilizing deep learning in dental X-ray images can help dentists quickly and accurately identify common dental diseases such as periodontitis and dental caries. This paper applies image processing and deep learning technologies to dental X-ray images to propose a simultaneous recognition method for periodontitis and dental caries. The single-tooth X-ray image is detected by the YOLOv7 object detection technique and cropped from the periapical X-ray image. Then, it is processed through contrast-limited adaptive histogram equalization to enhance the local contrast, and bilateral filtering to eliminate noise while preserving the edge. The deep learning architecture for classification comprises a pre-trained EfficientNet-B0 and fully connected layers that output two labels by the sigmoid activation function for the classification task. The average precision of tooth detection using YOLOv7 is 97.1%. For the recognition of periodontitis, the area under the curve (AUC) of the receiver operating characteristic (ROC) curve is 98.67%, and the AUC of the precision-recall (PR) curve is 98.38%. For the recognition of dental caries, the AUC of the ROC curve is 98.31%, and the AUC of the PR curve is 97.55%. Different from the conventional deep learning-based methods for a single disease such as periodontitis or dental caries, the proposed approach can provide the recognition of both periodontitis and dental caries simultaneously. This recognition method presents good performance in the identification of periodontitis and dental caries, thus facilitating dental diagnosis.

Detection of Anomalous Behavior of Manufacturing Workers Using Deep Learning-Based Recognition of Human–Object Interaction

The increasing demand for industrial products has expanded production quantities, leading to negative effects on product quality, worker productivity, and safety during working hours. Therefore, monitoring the conditions in manufacturing environments, particularly human workers, is crucial. Accordingly, this study presents a model that detects workers’ anomalous behavior in manufacturing environments. The objective is to determine worker movements, postures, and interactions with surrounding objects based on human–object interactions using a Mask R-CNN, MediaPipe Holistic, a long short-term memory (LSTM), and worker behavior description algorithm. The process begins by recognizing the objects within video frames using a Mask R-CNN. Afterward, worker poses are recognized and classified based on object positions using a deep learning-based approach. Next, we identified the patterns or characteristics that signified normal or anomalous behavior. In this case, anomalous behavior consists of anomalies correlated with human pose recognition (emergencies: worker falls, slips, or becomes ill) and human pose recognition with object positions (tool breakage and machine failure). The findings suggest that the model successfully distinguished anomalous behavior and attained the highest pose recognition accuracy (approximately 96%) for standing, touching, and holding, and the lowest accuracy (approximately 88%) for sitting. In addition, the model achieved an object detection accuracy of approximately 97%.

Deep Learning-based Recognition of Devanagari Handwritten Characters

Numerous techniques have been used over many years to study handwriting recognition. There are two methods for reading handwriting, one of which is online and the other offline. Image recognition is the main part of the handwriting recognition process. Image recognition gives careful consideration to the picture's dimensions, viewing angle, and image quality. Machine learning and deep learning techniques are the two areas of focus for developers looking to increase the intelligence of computers. A person may learn to perform a task by repeatedly exercising it until they recall how to do it. His brain's neurons begin to work automatically, enabling him to carry out the task he has quickly learned. This and deep learning are fairly similar. It uses a variety of neural network designs to address a range of problems. The convolution neural network (CNN) is a very effective technique for handwriting and picture detection.