Multiple Deep Learning Models Research Articles

An analysis of decipherable red blood cell abnormality detection under federated environment leveraging XAI incorporated deep learning

In recent times, automated detection of diseases from pathological images leveraging Machine Learning (ML) models has become fairly common, where the ML models learn detecting the disease by identifying biomarkers from the images. However, such an approach requires the models to be trained on a vast amount of data, and healthcare organizations often tend to limit access due to privacy concerns. Consequently, collecting data for traditional centralized training becomes challenging. These privacy concerns can be handled by Federation Learning (FL), which builds an unbiased global model from local models trained with client data while maintaining the confidentiality of local data. Using FL, this study solves the problem of centralized data collection by detecting deformations in images of Red Blood Cells (RBC) in a decentralized way. To achieve this, RBC data is used to train multiple Deep Learning (DL) models, and among the various DL models, the most efficient one is considered to be used as the global model inside the FL framework. The FL framework works by copying the global model’s weight to the client’s local models and then training the local models in client-specific devices to average the weights of the local model back to the global model. In the averaging process, direct averaging is performed and alongside, weighted averaging is also done to weigh the individual local model’s contribution according to their performance, keeping the FL framework immune to the effects of bad clients and attacks. In the process, the data of the client remains confidential during training, while the global model learns necessary information. The results of the experiments indicate that there is no significant difference in the performance of the FL method and the best-performing DL model, as the best-performing DL model reaches an accuracy of 96% and the FL environment reaches 94%-95%. This study shows that the FL technique, in comparison to the classic DL methodology, can accomplish confidentiality secured RBC deformation classification from RBC images without substantially diminishing the accuracy of the categorization. Finally, the overall validity of the classification results has been verified by employing GradCam driven Explainable AI techniques.

Achieving enhanced diagnostic precision in endometrial lesion analysis through a data enhancement framework

ObjectiveThe aim of this study was to enhance the precision of categorization of endometrial lesions in ultrasound images via a data enhancement framework based on deep learning (DL), through addressing diagnostic accuracy challenges, contributing to future research.Materials and methodsUltrasound image datasets from 734 patients across six hospitals were collected. A data enhancement framework, including image features cleaning and soften label, was devised and validated across multiple DL models, including ResNet50, DenseNet169, DenseNet201, and ViT-B. A hybrid model, integrating convolutional neural network and transformer architectures for optimal performance, to predict lesion types was developed.ResultsImplementation of our novel strategies resulted in a substantial enhancement in model accuracy. The ensemble model achieved accuracy and macro-area under the receiver operating characteristic curve values of 0.809 of 0.911, respectively, underscoring the potential for use of DL in endometrial lesion ultrasound image classification.ConclusionWe successfully developed a data enhancement framework to accurately classify endometrial lesions in ultrasound images. Integration of anomaly detection, data cleaning, and soften label strategies enhanced the comprehension of lesion image features by the model, thereby boosting its classification capacity. Our research offers valuable insights for future studies and lays the foundation for creation of more precise diagnostic tools.

An adversarial diverse deep ensemble approach for surrogate‐based traffic signal optimization

AbstractSurrogate‐based traffic signal optimization (TSO) is a computationally efficient alternative to simulation‐based TSO. By replacing the simulation‐based objective function, a surrogate model can quickly identify solutions by searching for extreme points on its response surface. As a popular surrogate model, the ensemble of multiple diverse deep learning models can approximate complicated systems with a strong generalizability. However, existing ensemble methods barely focus on strengthening the prediction of extreme points, which we found can be realized by further diversifying base learners in the ensemble. The study proposes an adversarial diverse ensemble (ADE) method for online TSO with limited computational resources, comprising two stages: In the offline stage, base extractors are diversified with unlabeled data by a designed adversarial diversity training algorithm; in the online stage, base predictors are trained in parallel with limited labeled data, and the ensemble then serves as the surrogate model to search for solutions iteratively for TSO. First, it is demonstrated that the prediction accuracy on extreme points, and associated solution quality, can be constantly improved with base learners’ diversity enhanced by ADE. Case studies of TSO conducted on a four‐intersection arterial further demonstrate the superior solution quality and computational efficiency of the ADE surrogate model in a wide range of traffic scenarios. Moreover, a large‐scale online TSO experiment under dynamic traffic demand proves ADE's effectiveness in practical applications.

AI-Generated Spam Review Detection Framework with Deep Learning Algorithms and Natural Language Processing

Spam reviews pose a significant challenge to the integrity of online platforms, misleading consumers and undermining the credibility of genuine feedback. This paper introduces an innovative AI-generated spam review detection framework that leverages Deep Learning algorithms and Natural Language Processing (NLP) techniques to identify and mitigate spam reviews effectively. Our framework utilizes multiple Deep Learning models, including Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, Gated Recurrent Unit (GRU), and Bidirectional LSTM (BiLSTM), to capture intricate patterns in textual data. The system processes and analyzes large volumes of review content to detect deceptive patterns by utilizing advanced NLP and text embedding techniques such as One-Hot Encoding, Word2Vec, and Term Frequency-Inverse Document Frequency (TF-IDF). By combining three embedding techniques with four Deep Learning algorithms, a total of twelve exhaustive experiments were conducted to detect AI-generated spam reviews. The experimental results demonstrate that our approach outperforms the traditional machine learning models, offering a robust solution for ensuring the authenticity of online reviews. Among the models evaluated, those employing Word2Vec embeddings, particularly the BiLSTM_Word2Vec model, exhibited the strongest performance. The BiLSTM model with Word2Vec achieved the highest performance, with an exceptional accuracy of 98.46%, a precision of 0.98, a recall of 0.97, and an F1-score of 0.98, reflecting a near-perfect balance between precision and recall. Its high F2-score (0.9810) and F0.5-score (0.9857) further highlight its effectiveness in accurately detecting AI-generated spam while minimizing false positives, making it the most reliable option for this task. Similarly, the Word2Vec-based LSTM model also performed exceptionally well, with an accuracy of 97.58%, a precision of 0.97, a recall of 0.96, and an F1-score of 0.97. The CNN model with Word2Vec similarly delivered strong results, achieving an accuracy of 97.61%, a precision of 0.97, a recall of 0.96, and an F1-score of 0.97. This study is unique in its focus on detecting spam reviews specifically generated by AI-based tools rather than solely detecting spam reviews or AI-generated text. This research contributes to the field of spam detection by offering a scalable, efficient, and accurate framework that can be integrated into various online platforms, enhancing user trust and the decision-making processes.

A Performance Comparison Study on Climate Prediction in Weifang City Using Different Deep Learning Models

Climate change affects the water cycle, water resource management, and sustainable socio-economic development. In order to accurately predict climate change in Weifang City, China, this study utilizes multiple data-driven deep learning models. The climate data for 73 years include monthly average air temperature (MAAT), monthly average minimum air temperature (MAMINAT), monthly average maximum air temperature (MAMAXAT), and monthly total precipitation (MP). The different deep learning models include artificial neural network (ANN), recurrent NN (RNN), gate recurrent unit (GRU), long short-term memory neural network (LSTM), deep convolutional NN (CNN), hybrid CNN-GRU, hybrid CNN-LSTM, and hybrid CNN-LSTM-GRU. The CNN-LSTM-GRU for MAAT prediction is the best-performing model compared to other deep learning models with the highest correlation coefficient (R = 0.9879) and lowest root mean square error (RMSE = 1.5347) and mean absolute error (MAE = 1.1830). These results indicate that The hybrid CNN-LSTM-GRU method is a suitable climate prediction model. This deep learning method can also be used for surface water modeling. Climate prediction will help with flood control and water resource management.

Automatic Detection and Identification of Calcareous Nannofossils in Chalk Using Deep Learning: A Proof-of-Concept Study for Biostratigraphy and Climate Research

Calcareous nannofossils serve as crucial indicators for establishing the biostratigraphic age of chalk macrofossil specimens in natural science collections. Better age control of specimens collected several hundred years ago enables us to uncover the dark data hidden within these collections and incorporate these data into current research projects, examining ecosystem response to past climate change. However, the manual identification of these microscopic organisms is laborious and subjective, and so we are harnessing deep learning techniques for automatic nannofossil detection and identification. This approach required the construction of a robust dataset, currently comprising over 100,000 labelled images, complemented by the development of multiple specialised deep learning models. While some models focus on detecting target species, others are dedicated to species classification. Evaluation on an independent test set showcases the efficacy of our methodology, with the current detection model achieving a balanced accuracy of 93%. Similarly, the classification model demonstrates robust performance, attaining an average balanced accuracy of 96%. Furthermore, as well as assisting with our biostratigraphic studies, the dataset of accurately labelled images has enabled us to test other aspects of ecosystem response. For example, examining morphometric changes in nannofossils over geological time can provide valuable insights into the potential impact of current global warming on modern phytoplankton assemblages (Mancini et al. 2021). This is particularly important for coccolithophores (Young et al. 2005), which play a critical role as primary producers in the global carbon cycle. A decrease in their size could lead to bottom-up ecosystem impacts and reduced carbon sequestration (Poulton et al. 2007, Krumhardt et al. 2017). Using our dataset, we conducted a deep-learning-enhanced automatic morphometric analysis focusing on the nannofossil species, Tranolithus orionatus. Our analysis revealed that two key morphometric parameters, minor axis and area size, showed statistically significant differences between the Cenomanian stage (approximately 100.5 to 93.9 million years ago) and the post-Cenomanian stages of the Late Cretaceous (approximately 93.9 to 66.0 million years ago). Kolmogorov-Smirnov tests (Massey 1951) between the two samples yielded p-values of 0.039 for the minor axis and 0.031 for the area size. Understanding these morphometric changes is crucial due to the close parallels between current climate projections and the warming and greenhouse climate of the Late Cretaceous, particularly the Cenomanian-Turonian boundary event (Arthur et al. 1990). Insights into how organisms changed morphologically during past periods of environmental stress can help us more effectively predict future responses of organisms under similar conditions (Razmjooei et al. 2020). These findings underscore the effectiveness of our approach in automating the identification and recognition of chalk nannofossils, helping to unlock natural science collections and to address key questions related to marine response to past climate change.

Improving predictions of rock tunnel squeezing with ensemble Q-learning and online Markov chain

Predicting rock tunnel squeezing in underground projects is challenging due to its intricate and unpredictable nature. This study proposes an innovative approach to enhance the accuracy and reliability of tunnel squeezing prediction. The proposed method combines ensemble learning techniques with Q-learning and online Markov chain integration. A deep learning model is trained on a comprehensive database comprising tunnel parameters including diameter (D), burial depth (H), support stiffness (K), and tunneling quality index (Q). Multiple deep learning models are trained concurrently, leveraging ensemble learning to capture diverse patterns and improve prediction performance. Integration of the Q-learning-Online Markov Chain further refines predictions. The online Markov chain analyzes historical sequences of tunnel parameters and squeezing class transitions, establishing transition probabilities between different squeezing classes. The Q-learning algorithm optimizes decision-making by learning the optimal policy for transitioning between tunnel states. The proposed model is evaluated using a dataset from various tunnel construction projects, assessing performance through metrics like accuracy, precision, recall, and F1-score. Results demonstrate the efficiency of the ensemble deep learning model combined with Q-learning-Online Markov Chain in predicting surrounding rock tunnel squeezing. This approach offers insights into parameter interrelationships and dynamic squeezing characteristics, enabling proactive planning and support measures implementation to mitigate tunnel squeezing hazards and ensure underground structure safety. Experimental results show the model achieves a prediction accuracy of 98.11%, surpassing individual CNN and RNN models, with an AUC value of 0.98.

A new methodology for establishing an SOC content prediction model that is spatiotemporally transferable at multidecadal and intercontinental scales

Quantifying and tracking the soil organic carbon (SOC) content is a key step toward long-term terrestrial ecosystem monitoring. Over the past decade, numerous models have been proposed and have achieved promising results for predicting SOC content. However, many of these studies are confined to specific temporal or spatial contexts, neglecting model transferability. Temporal transferability refers to a model’s ability to be applied across different periods, while spatial transferability relates to its applicability across diverse geographic locations for prediction. Therefore, developing a new methodology to establish a prediction model with high spatiotemporal transferability for SOC content is critically important. In this study, two large intercontinental study areas were selected, and measured topsoil (0–20 cm) sample data, 27,059 cloudless Landsat 5/8 images, digital elevation models, and climate data were acquired for 3 periods. Based on these data, monthly average climate data, monthly average data reflecting soil properties, and topography data were calculated as original input (OI) variables. We established an innovative multivariate deep learning model with high spatiotemporal transferability, combining the advantages of attention mechanism, graph neural network, and long short-term memory network model (A-GNN-LSTM). Additionally, the spatiotemporal transferability of A-GNN-LSTM and commonly used prediction models were compared. Finally, the abilities of the OI variables and the OI variables processed by feature engineering (FEI) for different SOC prediction models were explored. The results show that 1) the A-GNN-LSTM that used OI as the input variable was the optimal prediction model (RMSE = 4.86 g kg−1, R2 = 0.81, RPIQ = 2.46, and MAE = 3.78 g kg−1) with the highest spatiotemporal transferability. 2) Compared to the temporal transferability of the GNN, the A-GNN-LSTM demonstrates superior temporal transferability (ΔR2T = −0.10 vs. −0.07). Furthermore, compared to the spatial transferability of LSTM, the A-GNN-LSTM shows enhanced spatial transferability (ΔR2S = −0.16 vs. −0.09). These findings strongly suggest that the fusion of geospatial context and temporally dependent information, extracted through the integration of GNN and LSTM models, effectively enhances the spatiotemporal transferability of the models. 3) By introducing the attention mechanism, the weights of different input variables could be calculated, increasing the physical interpretability of the deep learning model. The largest weight was assigned to climate data (39.55 %), and the smallest weight was assigned to vegetation (19.96 %). 4) Among the commonly used prediction models, the deep learning model had higher prediction accuracy (RMSE = 6.64 g kg−1, R2 = 0.64, RPIQ = 1.78, and MAE = 4.78 g kg−1) and spatial transferability (ΔRMSES = 1.43 g kg−1, ΔR2S = −0.13, ΔRPIQS = −0.50, and ΔMAES = 1.09 g kg−1), and the linear model had the higher temporal transferability (ΔRMSET = 1.46 g kg−1, ΔR2T = −0.14, ΔRPIQT = −0.45, and ΔMAET = 1.29 g kg−1). 5) The deep learning models necessitated the OI, whereas the linear and traditional machine learning models necessitated FEI to achieve higher prediction accuracy. This study presents an important step forward in integrating multiple deep learning models to build a highly spatiotemporal transferability SOC prediction model.

Deep Learning to Predict the Future Growth of Geographic Atrophy from Fundus Autofluorescence

PurposeThe region of growth (ROG) of geographic atrophy (GA) throughout the macular area has an impact on visual outcomes. Here, we developed multiple deep learning (DL) models to predict the 1-year ROG of GA lesions using fundus autofluorescence (FAF) images. DesignIn this retrospective analysis, 3 types of models were developed using FAF images collected 6 months after baseline to predict the GA lesion area (segmented lesion mask) at 1.5 years, FAF images collected at baseline and 6 months to predict the GA lesion at 1.5 years, and FAF images collected 6 months after baseline to predict the GA lesion at 1 and 1.5 years. The 1-year ROG from the 6-month visit was derived by taking the difference between the GA lesion area (segmented lesion mask) at the 1.5-year and 6-month visits. ParticipantsPatients enrolled in the following lampalizumab clinical trials and prospective observational studies: NCT02247479, NCT02247531, NCT02479386, and NCT02399072. MethodsDatasets of study eyes from 597 patients were split into model training (310), validation (78), and test sets (209), stratified by baseline or initial lesion area, lesion growth rate, foveal involvement, and focality. DL experiments were performed using the 2-dimensional U-Net, whole-lesion and multiclass models were developed. Main Outcome MeasuresThe performance of the models was evaluated by calculating the Dice score, coefficient of determination (R2), and the squared Pearson correlation coefficient (r2) between the true and derived GA lesion 1-year ROG. ResultsThe model using baseline and 6-month FAF images to predict GA lesion enlargement at 1.5 years had the best performance for the derived 1-year ROG. Mean Dice scores were 0.73, 0.68, and 0.70, respectively, in the training, validation, and test sets. The R2 (0.77, 0.53, and 0.79) and r2 (0.83, 0.61, and 0.79) showed similar trends across the 3 sets. ConclusionsThese findings show the potential of using baseline and/or 6-month visit FAF images to predict 1-year GA ROG using a DL approach. This work could potentially help support decision making in clinical trials and more informed treatment decisions in clinical practice.

Android malware defense through a hybrid multi-modal approach

The rapid proliferation of Android apps has given rise to a dark side, where increasingly sophisticated malware poses a formidable challenge for detection. To combat this evolving threat, we present an explainable hybrid multi-modal framework. This framework leverages the power of deep learning, with a novel model fusion technique, to illuminate the hidden characteristics of malicious apps. Our approach combines models (leveraging late fusion approach) trained on attributes derived from static and dynamic analysis, hence utilizing the unique strengths of each model. We thoroughly analyze individual feature categories, feature ensembles, and model fusion using traditional machine learning classifiers and deep neural networks across diverse datasets. Our hybrid fused model outperforms others, achieving an F1-score of 99.97% on CICMaldroid2020. We use SHAP (SHapley Additive exPlanations) and t-SNE (t-distributed Stochastic Neighbor Embedding) to further analyze and interpret the best-performing model. We highlight the efficacy of our architectural design through an ablation study, revealing that our approach consistently achieves over 99% detection accuracy across multiple deep learning models. This paves the way groundwork for substantial advancements in security and risk mitigation within interconnected Android OS environments.

Development of Machine Learning and Deep Learning Prediction Models for PM2.5 in Ho Chi Minh City, Vietnam

The application of machine learning and deep learning in air pollution management is becoming increasingly crucial, as these technologies enhance the accuracy of pollution prediction models, facilitating timely interventions and policy adjustments. They also facilitate the analysis of large datasets to identify pollution sources and trends, ultimately contributing to more effective and targeted environmental protection strategies. Ho Chi Minh City (HCMC), a major metropolitan area in southern Vietnam, has experienced a significant rise in air pollution levels, particularly PM2.5, in recent years, creating substantial risks to both public health and the environment. Given the challenges posed by air quality issues, it is essential to develop robust methodologies for predicting PM2.5 concentrations in HCMC. This study seeks to develop and evaluate multiple machine learning and deep learning models for predicting PM2.5 concentrations in HCMC, Vietnam, utilizing PM2.5 and meteorological data over 911 days, from 1 January 2021 to 30 June 2023. Six algorithms were applied: random forest (RF), extreme gradient boosting (XGB), support vector regression (SVR), artificial neural network (ANN), generalized regression neural network (GRNN), and convolutional neural network (CNN). The results indicated that the ANN is the most effective algorithm for predicting PM2.5 concentrations, with an index of agreement (IOA) value of 0.736 and the lowest prediction errors during the testing phase. These findings imply that the ANN algorithm could serve as an effective tool for predicting PM2.5 concentrations in urban environments, particularly in HCMC. This study provides valuable insights into the factors that affect PM2.5 concentrations in HCMC and emphasizes the capacity of AI methodologies in reducing atmospheric pollution. Additionally, it offers valuable insights for policymakers and health officials to implement targeted interventions aimed at reducing air pollution and improving public health.

Enabling Intelligence on the Edge: Leveraging Edge Impulse to Deploy Multiple Deep Learning Models on Edge Devices for Tomato Leaf Disease Detection

Tomato diseases, including Leaf blight, Leaf curl, Septoria leaf spot, and Verticillium wilt, are responsible for up to 50% of annual yield loss, significantly impacting global tomato production, valued at approximately USD 87 billion. In Ghana, there is a yield gap of about 50% in tomato production, which requires drastic measures to increase the yield of tomatoes. Conventional diagnostic methods are labor-intensive and impractical for real-time application, highlighting the need for innovative solutions. This study addresses these issues in Ghana by utilizing Edge Impulse to deploy multiple deep-learning models on a single mobile device, facilitating the rapid and precise detection of tomato leaf diseases in the field. This work compiled and rigorously prepared a comprehensive Ghanaian dataset of tomato leaf images, applying advanced preprocessing and augmentation techniques to enhance robustness. Using TensorFlow, we designed and optimized efficient convolutional neural network (CNN) architectures, including MobileNet, Inception, ShuffleNet, Squeezenet, EfficientNet, and a custom Deep Neural Network (DNN). The models were converted to TensorFlow Lite format and quantized to int8, substantially reducing the model size and improving inference speed. Deployment files were generated, and the Edge Impulse platform was configured to enable multiple model deployments on a mobile device. Performance evaluations across edge hardware provided metrics such as inference speed, accuracy, and resource utilization, demonstrating reliable real-time detection. EfficientNet achieved a high training accuracy of 97.12% with a compact 4.60 MB model size, proving its efficacy for mobile device deployment. In contrast, the custom DNN model is optimized for microcontroller unit (MCU) deployment. This edge artificial intelligence (AI) technology integration into agricultural practices offers scalable, cost-effective, and accessible solutions for disease classification, enhancing crop management, and supporting sustainable farming practices.

Evaluating fine tuned deep learning models for real-time earthquake damage assessment with drone-based images

Earthquakes pose a significant threat to life and property worldwide. Rapid and accurate assessment of earthquake damage is crucial for effective disaster response efforts. This study investigates the feasibility of employing deep learning models for damage detection using drone imagery. We explore the adaptation of models like VGG16 for object detection through transfer learning and compare their performance to established object detection architectures like YOLOv8 (You Only Look Once) and Detectron2. Our evaluation, based on various metrics including mAP, mAP50, and recall, demonstrates the superior performance of YOLOv8 in detecting damaged buildings within drone imagery, particularly for cases with moderate bounding box overlap. This finding suggests its potential suitability for real-world applications due to the balance between accuracy and efficiency. Furthermore, to enhance real-world feasibility, we explore two strategies for enabling the simultaneous operation of multiple deep learning models for video processing: frame splitting and threading. In addition, we optimize model size and computational complexity to facilitate real-time processing on resource-constrained platforms, such as drones. This work contributes to the field of earthquake damage detection by (1) demonstrating the effectiveness of deep learning models, including adapted architectures, for damage detection from drone imagery, (2) highlighting the importance of evaluation metrics like mAP50 for tasks with moderate bounding box overlap requirements, and (3) proposing methods for ensemble model processing and model optimization to enhance real-world feasibility. The potential for real-time damage assessment using drone-based deep learning models offers significant advantages for disaster response by enabling rapid information gathering to support resource allocation, rescue efforts, and recovery operations in the aftermath of earthquakes.

Deep learning-based temperature prediction during rotary ultrasonic bone drilling

Bone drilling is a common but critical medical procedure in orthopedic surgeries used to treat fractured bones. During this procedure, the temperature of the bone increases due to generation of frictional energy. Temperature control has been a major challenge in bone drilling since its foundation. If this temperature increases over 47°C for 1 min, then it can result in permanent bone damage. To control the temperature elevation this study proposes a deep learning-based robust predictive model which has been trained and tested on data from pig bones. Excessive in-house testing has been done on pig femur bones to gather data and verify the results. Rotary ultrasonic bone drilling was the machining process used for drilling. Four independent variables which were rotational speed, feed rate, abrasive grit size, and vibrational ultrasonic power were varied and the temperature for each set of values was recorded. Multiple deep learning models were made and were compared on different error metrics. It was found that convolutional neural network 1D gave the least error over other models. The error generated by deep learning models was less than mathematical and experimental models.

Shapley visual transformers for image-to-text generation

In the contemporary landscape of the web, text-to-image generation stands out as a crucial information service. Recently, deep learning has emerged as the cutting-edge methodology for advancing text-to-image generation systems. However, these models are typically constructed using domain knowledge specific to the application at hand and a very particular data distribution. Consequently, data scientists must be well-versed in the relevant subject. In this research work, we target a new foundation for text-to-image generation systems by introducing a consensus method that facilitates self-adaptation and flexibility to handle different learning tasks and diverse data distributions. This paper presents I2T-SP (Image-to-Text Generation for Shapley Pruning) as a consensus method for general-purpose intelligence without the assistance of a domain expert. The trained model is developed using a general deep-learning approach that investigates the contribution of each model in the training process. Multiple deep learning models are trained for each set of historical data, and the Shapley Value is determined to compute the contribution of each subset of models in the training. Subsequently, the models are pruned according to their contribution to the learning process. We present the evaluation of the generality of I2T-SP using different datasets with varying shapes and complexities. The results reveal the effectiveness of I2T-SP compared to baseline image-to-text generation solutions. This research marks a significant step towards establishing a more adaptable and broadly applicable foundation for image-to-text generation systems.

Towards a Deep Learning-Powered Herbarium Image Analysis Platform

Global digitization efforts have archived millions of specimen scans worldwide in herbarium collections, which are essential for studying plant evolution and biodiversity. ReColNat hosts, at present, over 10 million images. However, analyzing these datasets poses crucial challenges for botanical research. The application of deep learning in biodiversity analyses, particularly in analyzing herbarium scans, has shown promising results across numerous tasks (Ariouat et al. 2023, Ariouat et al. 2024, Groom et al. 2023, Sahraoui et al. 2023). Within the e-Col+project (ANR-21-ESRE-0053), we are developing multiple deep learning models aimed at identifying plant morphological traits. We have developed pipelines and models for cleaning, analyzing, and transforming herbarium images, including models for: i) detecting non-vegetal elements, such as barcodes, envelopes, labels, etc.; ii) detecting plant organs, including leaves, flowers, fruits, etc.; and iii) segmenting to recognize plant parts for image cleaning. We are also developing models for classification tasks related to various morphological traits. To validate these models, improve their generalization, and make them easily usable by end-users, deploying them within a generic platform is crucial. The generic platform called PlantAI, currently under development by the e-Col+ project, should enable easy deployment during development for testing and allow users to load annotations for new traits in order to train a model and add it to the existing catalog. The platform is based on a microservice architecture, allowing users to upload images, create custom datasets, and access various AI models for image analysis. The platform is composed of four main modules, as illustrated in Fig. 1. The first module is the collaborative workspace manager, which allows users to create projects and image datasets and invite other users to collaborate on a project. The second module is the navigation interface and dashboards. This module integrates a search engine using metadata and AI annotations, a navigation interface between projects, datasets, and specimens, as well as dashboards for analysis across datasets, specimens, and AI models. The third module is the dataset manager, which handles metadata and annotations associated with the specimens. These annotations can be produced either by expert users or by AI models. The fourth module is the AI models management module, so that models can be used to generate AI annotations of specimen. During the development lifecycle of an AI model, users can create datasets and annotate them with AI models. These annotations can be in two possible states: validated by experts and non-validated. Users collaborating on a project can indicate errors in the model predictions and leave comments to explain their evaluations. These corrections made by experts can be used to retrain the models and thus improve their performance. This platform, will be highly beneficial for botanists, enhancing the efficiency and effectiveness of biodiversity analyses from herbarium scans. We aim to provide users with a catalog of AI models through this platform and allow them to import their own datasets with their own annotations regarding traits of their choice. Users will be able to select a model from the AI model catalog and train it using their dataset. Ultimately, the model obtained from this training will be automatically deployed to be available for AI annotation. The annotations produced by this model will be automatically available in the filtering and navigation interface, thus allowing for dynamic and automatic integration of the AI annotations into the navigation interface.

Advancing Prostate Cancer Diagnosis: A Deep Learning Approach for Enhanced Detection in MRI Images.

Prostate cancer remains a leading cause of mortality among men globally, necessitating advancements in diagnostic methodologies to improve detection and treatment outcomes. Magnetic Resonance Imaging has emerged as a crucial technique for the detection of prostate cancer, with current research focusing on the integration of deep learning frameworks to refine this diagnostic process. This study employs a comprehensive approach using multiple deep learning models, including a three-dimensional (3D) Convolutional Neural Network, a Residual Network, and an Inception Network to enhance the accuracy and robustness of prostate cancer detection. By leveraging the complementary strengths of these models through an ensemble method and soft voting technique, the study aims to achieve superior diagnostic performance. The proposed methodology demonstrates state-of-the-art results, with the ensemble model achieving an overall accuracy of 91.3%, a sensitivity of 90.2%, a specificity of 92.1%, a precision of 89.8%, and an F1 score of 90.0% when applied to MRI images from the SPIE-AAPM-NCI PROSTATEx dataset. Evaluation of the models involved meticulous pre-processing, data augmentation, and the use of advanced deep-learning architectures to analyze the whole MRI slices and volumes. The findings highlight the potential of using an ensemble approach to significantly improve prostate cancer diagnostics, offering a robust and precise tool for clinical applications.

Retrieval performance of mangrove tree heights using multiple machine learning regression models and UAV-LiDAR point clouds

ABSTRACT Mangroves are vital coastal ecosystems that provide crucial links between land and sea. Tree height is a key indicator for assessing mangroves’ health status. Currently, there are still numerous challenges in estimating mangrove tree height. In this study, multiple deep learning and shallow machine learning regression models were developed to accurately estimate mangrove tree height using multi-dimensional Light Detection and Ranging (LiDAR) point clouds and their derivatives. We constructed a novel CNN_RepMLP model for mangrove tree height mapping. We also further verified the applicability of different types of regression models in estimating mangrove tree heights, and explored the influence of different LiDAR-derived features on the inversion accuracy for tree heights. The results indicated the following: (1) The CNN_RepMLP displayed satisfactory performance in the inversion of mangrove tree heights, and exhibited better robustness and generalization ability than the convolutional neural network (CNN) model. (2) Among the different LiDAR-derived feature combinations, combining height variables with intensity variables can not only mitigate the negative impact of height variables on inversion models, but also enhance the mangrove tree height inversion accuracy. (3) The ensemble learning framework with ExtraTrees as the meta-model can make better use of the differences and complementarities between different single base models, and exhibited better accuracy in estimating the height of mangrove trees compared with other ensemble learning models. (4) Multiple machine learning models based on multi-dimensional UAV-LiDAR point-cloud-derived features are suitable for the inversion of mangrove tree height. The CNN_RepMLP model outperformed the CNN and stacking ensemble learning models and had more detailed differentiation in terms of mangrove tree height. Its prediction results can more realistically reflect the spatial differentiation characteristics of mangrove tree height.

Detection of road defects with weak small samples based on multiple deep learning models

Detection of road defects with weak small samples based on multiple deep learning models

CT-based artificial intelligence prediction model for ocular motility score of thyroid eye disease.

Thyroid eye disease (TED) is the most common orbital disease in adults. Ocular motility restriction is the primary complaint of patients, while its evaluation is quite difficult. The present study aimed to introduce an artificial intelligence (AI) model based on orbital computed tomography (CT) images for ocular motility score. A total of 410 sets of CT images and clinical data were obtained from the hospital. To build a triple classification predictive model for ocular motility score, multiple deep learning models were employed to extract features of images and clinical data. Subgroup analyses based on pertinent clinical features were performed to test the efficacy of models. The ResNet-34 network outperformed Alex-Net and VGG16-Net in prediction of ocular motility score, with the optimal accuracy (ACC) of 0.907, 0.870, and 0.890, respectively. Subgroup analyses indicated no significant difference in ACC between active or inactive phase, functional visual field diplopia or peripheral visual field diplopia (p > 0.05). However, in the gender subgroup, the prediction model performed more accurately in female patients than males (p = 0.02). In conclusion, the AI model based on CT images and clinical data successfully realized automatic scoring of ocular motility in TED patients. This approach potentially enhanced the efficiency and accuracy of ocular motility evaluation, thus facilitating clinical application.