Average Precision Research Articles

Development of Machine Learning Copilot to Assist Novices in Learning Flexible Laryngoscopy.

Here we describe the development and pilot testing of the first artificial intelligence (AI) software "copilot" to help train novices to competently perform flexible fiberoptic laryngoscopy (FFL) on a mannikin and improve their uptake of FFL skills. Supervised machine learning was used to develop an image classifier model, dubbed the "anatomical region classifier," responsible for predicting the location of camera in the upper aerodigestive tract and an object detection model, dubbed the "anatomical structure detector," responsible for locating and identifying key anatomical structures in images. Training data were collected by performing FFL on an AirSim Combo Bronchi X mannikin (United Kingdom, TruCorp Ltd) using an Ambu aScope 4 RhinoLaryngo Slim connected to an Ambu® aView™ 2 Advance Displaying Unit (Ballerup, Ambu A/S). Medical students were prospectively recruited to try the FFL copilot and rate its ease of use and self-rate their skills with and without the copilot. This model classified anatomical regions with an overall accuracy of 91.9% on the validation set and 80.1% on the test set. The model detected anatomical structures with overall mean average precision of 0.642. Through various optimizations, we were able to run the AI copilot at approximately 28 frames per second (FPS), which is imperceptible from real time and nearly matches the video frame rate of 30 FPS. Sixty-four novice medical students were recruited for feedback on the copilot. Although 90.9% strongly agreed/agreed that the AI copilot was easy to use, their self-rating of FFL skills following use of the copilot were overall equivocal to their self-rating without the copilot. The AI copilot tracked successful capture of diagnosable views of key anatomical structures effectively guiding users through FFL to ensure all anatomical structures are sufficiently captured. This tool has the potential to assist novices in efficiently gaining competence in FFL. NA Laryngoscope, 2024.

A-317 Development of a patient-focused volumetric absorptive microsampling strategy to assess urinary amino acid excretion

Abstract Background Biochemical determinations of urinary amino acid excretion patterns can diagnose specific inborn errors of metabolism (IEM). Clinical management of these genetic metabolic patients may require frequent urine amino acid tests. A simplified process for transporting home collected urine specimens would benefit the IEM community. Volumetric absorptive microsampling (VAMS) is an emergent alternative sampling technique for the collection of dried biological specimens. We describe the development and preliminary validation results of a VAMS strategy for the analysis of creatinine and amino acids (N=22) from microsampled dried urine. Methods Urine creatinine and amino acids were respectively measured in a clinical laboratory setting using established assays on a CobasPro (Roche) and a MassTrak ultra-performance liquid chromatography (UPLC) amino acid analysis (AAA) solution (Waters). Two 30 µL VAMS tips from a Mitra device (Neoteryx) were used to sample each random urine specimen. An ultrasonication-based solvent extraction procedure was optimized to extract the amino acids and creatinine from the VAMS tips. Solvent extracts were subsequently dried by centrifugal evaporation. Extract from one tip was re-dissolved in water and tested for creatinine using an enzymatic method. The other extract was re-dissolved in 0.02 N sodium hydroxide and tested according to the MassTrak AAA procedure, which deploys pre-column AccQTag derivatization with reversed-phase UPLC on a C18 column (2.1 × 150 mm; 1.7 µm) and UV detection at 260 nm. Accuracy, within-batch precision and short-term stability of the optimized VAMS protocol were determined. Results Extraction solvent with a 60:40 (v/v) water:methanol mixture provided the highest average amino acid recovery. This extraction solvent produced higher average amino acid recoveries at pH 11 (89±13%), than pH 3 (82±10%) or pH 7 (72±11%). Creatinine measured from the VAMS tips (N=44) had a consistent negative bias of 20% relative to neat urine (R=0.9905, slope=0.802, intercept=0.02). The creatinine corrected urine amino acid results from the Mitra device were corrected for this observed concentration bias. The accuracy of the amino acid results derived from the VAMS tip vs neat urine was 104 ± 14%. The average within-batch precision (CV%) of the amino acids was 9.8% (range=6.7% (cystine) to 17.6% (arginine)) and 6.2% for creatinine. The average 1 week storage stability at ambient temperature was 91% (range=73% (ornithine) to 107% (glycine)) for the amino acids and 107% for creatinine. The devised VAMS strategy was applied to the analysis of microsampled dried urine from a patient with cystinuria (OMIM no. 220100). Marked elevations in arginine, cystine, lysine and ornithine were respectively observed and are consistent with the biochemical diagnosis of the patient. Conclusions Creatinine corrected amino acid results from the devised VAMS strategy exhibit good precision and accuracy when compared to tradition analyses of neat urine. The potential use of microsampled dried urine for the diagnosis and monitoring of selected IEM has been demonstrated. The analytical and clinical performance of this VAMS protocol must be further validated, including the use of microsampled dried urine from home specimen collections.

Predicting Multiple Outcomes Associated with Frailty based on Imbalanced Multi-label Classification.

Frailty syndrome is prevalent among the elderly, often linked to chronic diseases and resulting in various adverse health outcomes. Existing research has predominantly focused on predicting individual frailty-related outcomes. However, this paper takes a novel approach by framing frailty as a multi-label learning problem, aiming to predict multiple adverse outcomes simultaneously. In the context of multi-label classification, dealing with imbalanced label distribution poses inherent challenges to multi-label prediction. To address this issue, our study proposes a hybrid resampling approach tailored for handling imbalance problems in the multi-label scenario. The proposed resampling technique and prediction tasks were applied to a high-dimensional real-life medical dataset comprising individuals aged 65years and above. Several multi-label algorithms were employed in the experiment, and their performance was evaluated using multi-label metrics. The results obtained through our proposed approach revealed that the best-performing prediction model achieved an average precision score of 83%. These findings underscore the effectiveness of our method in predicting multiple frailty outcomes from a complex and imbalanced multi-label dataset.

Convolutional neural network for the early identification of weeds: A technological support to biodiversity and yield losses mitigation

Weeds are a major constraint for crop production and food security. Chemical management, the most utilized method for weed control, has serious drawbacks. In this context, the development of more sustainable methods like site-specific weed management (SSWM) is highly deemed. With this study, we assessed the possibility of applying two convolutional neural networks (CNN) for the recognition of different classes of weeds in a winter wheat (T. aestivum) cultivation based on RGB images. By using this methodology, we were able to recognize with a high average precision (AP > 0.6) some species whose abundance and distribution in the field was correlated with the final wheat yield. We demonstrated that where the presence of R. raphanistrum, A. arvensis and P. rhoeas was high at the tillering stage and weed biodiversity indexes were low (Menhinich's Index, Simpson's Reciprocal Index and Shannon's Index), wheat yield was significantly reduced. In contrast, higher weed biodiversity mitigated yield losses. Therefore, CNN is a useful tool for early evaluation of the impact that weeds may have on yield, and it can be used as SSWM classifier for an early mapping of weeds, which is critical to improve our understanding of weed ecology dynamics in agricultural fields.

Spatial distribution and fixed-precision sequential sampling plans for Popillia japonica (Coleoptera: Scarabaeidae) adults in primocane raspberry: influence of foliar insecticides.

The Japanese beetle, Popillia japonica Newman (Coleoptera: Scarabaeidae), an invasive species from northern Japan, was first detected in Minnesota in 1968. According to fruit growers and the Minnesota Department of Agriculture, population size and feeding damage has been an increasing concern since 2010. Based on trap-catch data, populations have recently exceeded 4,000 beetles/trap/week during July-August near raspberry fields, and can increase by an order of magnitude within 7-10 days. The primary goals of this study were to assess the spatial distribution of P. japonica adults in raspberry, and to develop and validate a practical fixed-precision sequential sampling plan for grower use. Taylor's Power Law (TPL) regression was used to characterize the beetle's spatial pattern in research plots and commercial fields, either with or without insecticide applications. We then used Green's plan to develop an enumerative sequential sampling plan to estimate P. japonica density in primocane raspberry. Beetle population data were collected at two locations in southern Minnesota, including the Rosemount Research and Outreach Center, and a commercial field near Forest Lake. The TPL results, via slope comparisons, indicated no significant differences in P. japonica spatial pattern between insecticide treated plots versus untreated plots, or among 4 different insecticides (P>0.05). Utilizing all spatial pattern data, we characterized the distribution of P. japonica beetles to be highly aggregated in raspberry, with TPL slopes ranging from b = 1.38 to 1.55; all slopes were found to be >1.0. Although the slopes were not significantly different, we accounted for variability in spatial pattern by using 33 independent data sets, and the Resampling for Validation of Sampling Plans (RVSP) model to validate a sampling plan with a final average precision level of 0.25 (SEM/mean), recommended for integrated pest management (IPM) purposes. The final sampling plan required an average sample number of only 15, 1-m-row samples, while providing high relative net precision (RNP), and thus a cost-effective, efficient sample plan for growers.

Automated localization of dike leakage outlets using UAV-borne thermography and YOLO-based object detectors

Leakage-induced soil erosion poses a major threat to dike failure, particularly during floods. Timely detection and notification of leakage outlets to dike management are crucial for ensuring dike safety. However, manual inspection, the current main approach for identifying leakage outlets, is costly, inefficient, and lacks spatial coverage. To achieve efficient and automatic localization of dike leakage outlets, an innovative strategy combining drones, infrared thermography, and deep learning is presented. Drones are employed for dikes’ surface sensing. Real-time images from these drones are sent to a server where well-trained detectors are deployed. Once a leakage outlet is detected, alarming information is remotely sent to dike managers. To realize this strategy, 4 thermal imagers were employed to image leaking outlets of several models and actual dikes. 9,231 hand-labeled thermal images with 13,387 leaking objects were selected for analysis. 19 detectors were trained using transfer learning. The best detector achieved a mean average precision of 95.8 % on the challenging test set. A full-scale embankment was constructed for leakage outlet detection tests. Various field tests confirmed the efficiency of the proposed leakage outlet localization method. In some tough conditions, the trained detector also evidently outperformed manual judgement. Results indicate that under typical circumstances, the localization error of the proposed method is within 5 m, demonstrating its practical reliability. Finally, the influencing factors and limits of the suggested strategy are thoroughly examined.

HeteroKGRep: Heterogeneous Knowledge Graph based Drug Repositioning

The process of developing new drugs is both time-consuming and costly, often taking over a decade and billions of dollars to obtain regulatory approval. Additionally, the complexity of patent protection for novel compounds presents challenges for pharmaceutical innovation. Drug repositioning offers an alternative strategy to uncover new therapeutic uses for existing medicines. Previous repositioning models have been limited by their reliance on homogeneous data sources, failing to leverage the rich information available in heterogeneous biomedical knowledge graphs. We propose HeteroKGRep, a novel drug repositioning model that utilizes heterogeneous graphs to address these limitations. HeteroKGRep is a multi-step framework that first generates a similarity graph from hierarchical concept relations. It then applies SMOTE over-sampling to address class imbalance before generating node sequences using a heterogeneous graph neural network. Drug and disease embeddings are extracted from the network and used for prediction. We evaluated HeteroKGRep on a graph containing biomedical concepts and relations from ontologies, pathways and literature. It achieved state-of-the-art performance with 99% accuracy, 95% AUC ROC and 94% average precision on predicting repurposing opportunities. Compared to existing homogeneous approaches, HeteroKGRep leverages diverse knowledge sources to enrich representation learning. Based on heterogeneous graphs, HeteroKGRep can discover new drug-disease associations, leveraging de novo drug development. This work establishes a promising new paradigm for knowledge-guided drug repositioning using multimodal biomedical data.

Uncertainty-Aware Deep Learning Characterization of Knee Radiographs for Large-Scale Registry Creation

We present an automated image ingestion pipeline for a knee radiography registry, integrating a multilabel image-semantic classifier with conformal prediction-based uncertainty quantification and an object detection model for knee hardware. Annotators retrospectively classified 26,000 knee images detailing presence, laterality, prostheses, and radiographic views. They further annotated surgical construct locations in 11,841 knee radiographs. An uncertainty-aware multilabel EfficientNet-based classifier was trained to identify the knee laterality, implants, and radiographic view. A classifier trained with embeddings from the EfficientNet model detected out-of-domain images. An object detection model was trained to identify 20 different knee implants. Model performance was assessed against a held-out internal and an external dataset using per-class F1 score, accuracy, sensitivity, and specificity. Conformal prediction was evaluated with marginal coverage and efficiency. Classification Model with Conformal Prediction: F1 scores for each label output > 0.98. Coverage of each label output was >0.99 and the average efficiency was 0.97. The F1 score was 0.99, with precision and recall for knee radiographs of 0.99. Mean average precision across all classes was 0.945 and ranged from 0.695 to 1.000. Average precision and recall across all classes were 0.950 and 0.886. We present a multilabel classifier with domain detection and an object detection model to characterize knee radiographs. Conformal prediction enhances transparency in cases when the model is uncertain.

YOLO-ACT: an adaptive cross-layer integration method for apple leaf disease detection.

Apple is a significant economic crop in China, and leaf diseases represent a major challenge to its growth and yield. To enhance the efficiency of disease detection, this paper proposes an Adaptive Cross-layer Integration Method for apple leaf disease detection. This approach, built upon the YOLOv8s architecture, incorporates three novel modules specifically designed to improve detection accuracy and mitigate the impact of environmental factors. Furthermore, the proposed method addresses challenges arising from large feature discrepancies and similar disease characteristics, ultimately improving the model's overall detection performance. Experimental results show that the proposed method achieves a mean Average Precision (mAP) of 85.1% for apple leaf disease detection, outperforming the latest state-of-the-art YOLOv10s model by 2.2%. Compared to the baseline, the method yields a 2.8% increase in mAP, with improvements of 5.1%, 3.3%, and 2% in Average Precision, Recall, and mAP50-95, respectively. This method demonstrates superiority over other classic detection algorithms. Notably, the model exhibits optimal performance in detecting Alternaria leaf spot, frog eye leaf spot, gray spot, powdery mildew, and rust, achieving mAPs of 84.3%, 90.4%, 80.8%, 75.7%, and 92.0%, respectively. These results highlight the model's ability to significantly reduce false negatives and false positives, thereby enhancing both detection and localization of diseases. This research offers a new theoretical foundation and direction for future advancements in apple leaf disease detection.

Nearshore optical video object detector based on temporal branch and spatial feature enhancement

The computing power of nearshore and ship-borne devices is limited, posing significant challenges for accurately detecting objects in real-time on such devices. We propose a nearshore video object detector (NVID) to tackle these challenges. Considering the abundance of dynamic entities in the nearshore environment, we have developed you can look more (YCLM) to perceive the temporal characteristics of these objects. Furthermore, to improve the ability to detect objects of different sizes of networks, we designed parallel deformable attention (PDA) based on the spatial features of objects. More importantly, we developed fast re-parameterization convolution (FREConv) and faster conv (FConv). Building on these innovations, we proposed a fast re-parameterization network (FRENet) specifically tailored to produce low-parameter, multi-scale feature outputs. With end-to-end training, our pipeline outperforms other state-of-the-art (SOTA) methods on the nearshore objects (NearshoreObjects) dataset (90.4 average precision (AP) 50 (＋4.7), 9.3 parameters (Params) (−1.0M), 24.8 frames per second (FPS) (Jetson Nano) (＋0.6)). In addition, NVID also achieved excellent results in the on board (OnBoard) dataset (90.3 AP50 (＋2.8), 9.3 params (−1.0M), 26.5 FPS (Jetson Nano) (＋0.8)). The source code can be accessed at https://github.com/Yuanlin-Zhao/NVID.

YOLO-Based Design and Optimization of Weld Seam Detection Model

Abstract The traditional welding seam inspection efficiency is low, the model possesses a significant amount of computational parameters, and it is not suitable for the application of small and medium-sized welding seam recognition machines. In view of the wide application and remarkable effect of deep learning in the domain of machine vision, a welding seam defect detection model designed by G-Efficientnet-CA neural network was proposed and optimized. EfficientNet was employed as the core architecture for extracting features to greatly reduce calculation parameters and model volume. The CA attention mechanism is incorporated to enhance the model’s capacity for focusing attention on the weld image, thereby improving its ability to discern and analyze relevant features, and the accuracy is improved. The Generalized Intersection over Union (GIoU) loss function is changed from the original loss function to optimize the calculation of the coincidence degree between the real frame and the predicted frame. The efficient K-Means++ clustering algorithm is utilized to calculate the initial anchor frame which is more suitable for different weld data sets. The experimental outcomes demonstrate that the optimized model, when compared to the YOLOv3 model, exhibits superior effectiveness in detecting the VOC2007 dataset, as evidenced by a notable 12.7% increase in the average precision (mAP) and a significant reduction of 88% in the number of parameters.

Deep learning neural network-assisted badminton movement recognition and physical fitness training optimization

This work aims to solve the problem of low accuracy in recognizing the trajectory of badminton movement. This work focuses on the visual system in badminton robots and conducts side detection and tracking of flying badminton in two-dimensional image plane video streams. Then, the cropped video images are input into a convolutional neural network frame by frame. By adding an attention mechanism, it helps identify the badminton movement trajectory. Finally, to address the detection challenge of flying badminton as a small target in video streams, the deep learning one-stage detection network, Tiny YOLOv2, is improved from both the loss function and network structure perspectives. Moreover, it is combined with the Unscented Kalman Filter algorithm to predict the trajectory of badminton movement. Simulation results show that the improved algorithm performs excellently in tracking and predicting badminton trajectories compared with the existing algorithms. The average accuracy of the proposed method for tracking badminton trajectories is 91.40 %, and the recall rate is 84.60 %. The average precision, recall, and frame rate of the measured trajectories in four simple and complex scenarios of badminton flight video streams are 96.7 %, 95.7 %, and 29.2 frames/second, respectively. They are all superior to other classic algorithms. It is evident that the proposed method can provide powerful support for badminton trajectory recognition and help improve the accuracy of badminton movement recognition.

A framework for automatically generating composite keywords for geo-tagged street images

Due to the ubiquity of sensor-equipped cameras such as smartphones, images are associated with spatial metadata, including camera’s geographical location and viewing orientation, which can be used for automatically generating better semantic keywords about geo-tagged urban street images in addition to visual keywords extracted from image analysis. This study introduces a novel framework for auto-tagging images that integrates both spatial and visual properties to generate comprehensive and accurate tags. The framework operates through four phases: extraction, abstraction, composition, and assessment. Our research highlights the benefits of combining visual and spatial analyses, demonstrated through a case study using geo-tagged urban street images from Orlando, Pittsburgh, and Manhattan. Experimental results show that the proposed framework significantly enhances the accuracy of keyword-based searches compared to conventional methods. In particular, based on our experiments, image search using the tags generated by our proposed framework, referred to as descriptive tags, achieved an average precision improvement factor of 0.9 compared to conventional tags. Additionally, our proposed ranking algorithm, which extends the term frequency-inverse document frequency (TF-IDF) algorithm, resulted in improvement factors of 0.86 for mean average precision (MAP) and 0.57 for mean reciprocal rank (MRR). Moreover, our framework’s flexibility and robustness make it suitable for diverse applications, from smart cities to online shopping. The paper also includes a detailed evaluation and user study, confirming the precision and reliability of the generated tags.

Feature-level Fusion vs. Score-level Fusion for Image Retrieval Based on Pre-trained Deep Neural Networks

Today’s complex multimedia content made retrieving images similar to the user’s query from the database a challenging task. The performance of a Content-Based Image Retrieval System (CBIR) system highly depends on the image representation in a form of low-level features and similarity measurement. The traditional visual descriptors that do not provide good prior domain knowledge could lead to poor performance retrieval results. On the other hand, Deep Convolutional Neural Networks (DCNNs) have recently achieved a remarkable success as methods for image classification in various domains. Recently, pre-trained deep convolution neural networks on thousands of classes have the ability to extract very accurate and representative features which, in addition to classification, can also be successfully used in image retrieval systems. ResNet152, GoogLeNet and InceptionV3 are some of the effective and successful examples of pre-trained DCNNs recently applied in a computer vision tasks such as object recognition, clustering, and classification. In this paper, two approaches for a CBIR system, namely early fusion and late fusion, have been presented and compared. The early fusion utilizes concatenation of the features extracted by each possible pair of DCNNs, that is ResNet152-GoogLeNet, ResNet152-InceptionV3, and GoogLeNet-InceptionV3, and the late fusion apply CombSum method with Z-Score standardization to combine the score results provided by each DCNN of the aforementioned pairs. In the experiments on a popular WANG dataset it has been shown that late fusion approach slightly outperforms early fusion approach. The best performance of our experiments in terms of Average Precision (AP) for the top 20 results reaches 96.82%.

Automatic detection of epileptic seizure based on one dimensional cascaded convolutional autoencoder with adaptive window-thresholding

Objective. Identifying the seizure occurrence period (SOP) in extended EEG recordings is crucial for neurologists to diagnose seizures effectively. However, many existing computer-aided diagnosis systems for epileptic seizure detection (ESD) primarily focus on distinguishing between ictal and interictal states in EEG recordings. This focus has limited their application in clinical settings, as these systems typically rely on supervised learning approaches that require labeled data.Approach. To address this, our study introduces an unsupervised learning framework for ESD using a 1D- cascaded convolutional autoencoder (1D-CasCAE). In this approach, EEG recordings from selected patients in the CHB-MIT datasets are first segmented into 5 s epochs. Eight informative channels are chosen based on the correlation coefficient and Shannon entropy. The 1D-CasCAE is designed to autonomously learn the characteristic patterns of interictal (non-seizure) segments through downsampling and upsampling processes. The integration of adaptive thresholding and a moving window significantly enhances the model's robustness, enabling it to accurately identify ictal segments in long EEG recordings.Main results. Experimental results demonstrate that the proposed 1D-CasCAE effectively learns normal EEG signal patterns and efficiently detects anomalies (ictal segments) using reconstruction errors. When compared with other leading methods in anomaly detection, our model exhibits superior performance, as evidenced by its average Gmean, sensitivity, specificity, precision, and false positive rate scores of 98.00% ± 3.51%, 94.94% ± 6.92%, 99.60% ± 0.30%, 79.92% ± 13.56% and 0.0044 ± 0.0030 h-1respectively for a typical patient in CHB-MIT datasets.Significance. The developed model framework can be employed in clinical settings, replacing the manual inspection process of EEG signals by neurologists. Furthermore, the proposed automated system can adapt to each patient's SOP through the use of variable time windows for seizure detection.

Application of a Question Answering System in the Ontology-Based Health Domain and Question Templates to Improve the Quality of Answers

Developing an effective Question Answering System (QAS) to address a wide range of health-related inquiries presents a significant challenge. This study introduces an innovative approach using an ontology-based QAS within the health domain, leveraging the Resource Description Framework (RDF) and SPARQL query language. By utilizing ontology, the system can more accurately map health concepts, improving the relevance and precision of its responses. Additionally, the incorporation of structured Question Templates enhances the system’s ability to understand and respond to diverse user queries. The system's performance was evaluated using Black Box Testing, which demonstrated that it consistently provides accurate and relevant answers. The system achieved a Mean Average Precision (MAP) of 79.6%, indicating its potential to effectively address a broad spectrum of health issues.

MSDAFL: molecular substructure-based dual attention feature learning framework for predicting drug-drug interactions.

Drug-drug interactions (DDIs) can cause unexpected adverse drug reactions, affecting treatment efficacy and patient safety. The need for computational methods to predict DDIs has been growing due to the necessity of identifying potential risks associated with drug combinations in advance. Although several deep learning methods have been recently proposed to predict DDIs, many overlook feature learning based on interactions between the substructures of drug pairs. In this work, we introduce a molecular Substructure-based Dual Attention Feature Learning framework (MSDAFL), designed to fully utilize the information between substructures of drug pairs to enhance the performance of DDI prediction. We employ a self-attention module to obtain a set number of self-attention vectors, which are associated with various substructural patterns of the drug molecule itself, while also extracting interaction vectors representing inter-substructure interactions between drugs through an interactive attention module. Subsequently, an interaction module based on cosine similarity is used to further capture the interactive characteristics between the self-attention vectors of drug pairs. We also perform normalization after the interaction feature extraction to mitigate overfitting. After applying three-fold cross-validation, the MSDAFL model achieved average precision scores of 0.9707, 0.9991, and 0.9987, and area under the receiver operating characteristic curve scores of 0.9874, 0.9934, and 0.9974 on three datasets, respectively. In addition, the experiment results of five-fold cross-validation and cross-datum study also indicate that MSDAFL performs well in predicting DDIs. Data and source codes are available at https://github.com/27167199/MSDAFL.

Transfer learning with YOLOV8 for real-time recognition system of American Sign Language Alphabet

Sign language serves as a sophisticated means of communication vital to individuals who are deaf or hard of hearing, relying on hand movements, facial expressions, and body language to convey nuanced meaning. American Sign Language (ASL) exemplifies this linguistic complexity with its distinct grammar and syntax. The advancement of real-time ASL gesture recognition has explored diverse methodologies, including motion sensors and computer vision techniques. This study specifically addresses the recognition of ASL alphabet gestures using computer vision through Mediapipe for hand movement tracking and YOLOv8 for training the deep learning model. The model achieved notable performance metrics: precision of 98%, recall rate of 98%, F1 score of 99%, mean Average Precision (mAP) of 98%, and mAP50-95 of 93%, underscoring its exceptional accuracy and sturdy capabilities.

An effective farmer-centred mobile intelligence solution using lightweight deep learning for integrated wheat pest management

Integrated Pest Management (IPM) techniques have been widely used in agriculture to manage pest damage in the most economical way and to minimise harm to people, property and the environment. However, current research and products on the market cannot consolidate this process. Most existing solutions either require experts to visually identify pests or cannot automatically assess pest levels and make decisions based on detection results. To make the process from pest identification to pest management decision making more automated and intelligent, we propose an end-to-end integrated pest management solution that uses deep learning for semi-automated pest detection and an expert system for pest management decision making. Specifically, a low computational cost sampling point generation algorithm is proposed to enable mobile devices to generate uniformly distributed sampling points in irregularly shaped fields. We build a pest detection model based on YoloX and use Pytorch Mobile to deploy it on mobile phones, allowing users to detect pests offline. We develop a standardised sampling specification and a mobile application to guide users to take photos that allow pest population density to be calculated. A rule-based expert system is established to derive pest management thresholds from prior agricultural knowledge and make decisions based on pest detection results. We also propose a human-in-the-loop algorithm to continuously track and update the validity of the thresholds in the expert system. The mean average precision of the pest detection model is 58.17% for 97 classes, 75.29% for 2 classes, and 57.33% for 11 classes on three pest datasets, respectively. The usability of the pest management system is assessed by the User Experience Surveys and achieves a System Usability Scale (SUS) score of 76. The usability of the proposed solution is validated by qualitative field experiments.

Federated Diverse Self-ensembling Learning Approach for Data Heterogenity in Drive Vision

Federated learning (FL) has developed as an efficient framework that can be used to train models across isolated data sources while also protecting the privacy of the data. In FL a common method is to construct local and global models together, with the global model (server) informing the local models and the local models (clients) updating the global model. Most present works assume clients have labeled datasets and the server has no data for supervised learning (SL) problems. In reality, clients lack the competence and drive to identify their data, while the server may host a tiny amount. How to reasonably use serverlabeled and client-unlabeled data is crucial in semi-supervised learning (SSL) and Cclientdata heterogeneity is widespread in FL. However, inadequate high-quality labels and non-IID client data, especially in autonomous driving, decrease model performance across domains and interact negatively. To solve this Semi-Supervised Federated Learning (SSFL) problem, we come up with a new FL algorithm called FedDSL in this work. We use self-ensemble learning and complementary negative learning in our method to make clients’ unsupervised learning on unlabeled data more accurate and efficient. It also coordinates the model training on both the server side and the clients’ side. In an important distinction to earlier research that kept some subset of labels at each client, our method is the first to implement SSFL for clients with 0% labeled non-IID data. Our contributions include the effectiveness of self-ensemble learning by using confidence score vector for calculating only for the current model performing data filtering and initiated negative learning by showing the data filtering performance in the beginning rounds. Our approach has been rigorously validated on two significant autonomous driving datasets, BDD100K and Cityscapes, proving to be highly effective. We have achieved state-of-the-art results and the metric that is utilized to evaluate the effectiveness of each detection task is mean average precision (mAP@0.5). Astonishingly FedDSL performs nearly as well as fully-supervised centralized training approaches, despite the fact that it only uses 25% of the labels in the Global model.