Model Training Research Articles

CNN based multi-label image classification for presentation recommender system

Recommender systems are super popular nowadays for giving personalized recommendations to users. The paper focuses on a special type of recommender system aimed to advise an improvement of a user’s presentation based on slides classification. Classification is a fundamental task in information extraction and retrieval with a goal to assign given resource to a predefined class. In some cases, as in slides’ categorization, resources can belong to multiple classes, leading to a multi-labelled classification problem. Two different approaches for solving a multi-labelled image classification problem for the purpose of a presentation recommender system are suggested and evaluated in the paper. They are based on problem transformation and algorithm adaptation strategies and utilize a convolutional neural network for model training.

GeoDTR+: Toward Generic Cross-View Geolocalization via Geometric Disentanglement.

Cross-View Geo-Localization (CVGL) estimates the location of a ground image by matching it to a geo-tagged aerial image in a database. Recent works achieve outstanding progress on CVGL benchmarks. However, existing methods still suffer from poor performance in cross-area evaluation, in which the training and testing data are captured from completely distinct areas. We attribute this deficiency to the lack of ability to extract the geometric layout of visual features and models' overfitting to low-level details. Our preliminary work (Zhang et al. 2022) introduced a Geometric Layout Extractor (GLE) to capture the geometric layout from input features. However, the previous GLE does not fully exploit information in the input feature. In this work, we propose GeoDTR+ with an enhanced GLE module that better models the correlations among visual features. To fully explore the LS techniques from our preliminary work, we further propose Contrastive Hard Samples Generation (CHSG) to facilitate model training. Extensive experiments show that GeoDTR+ achieves state-of-the-art (SOTA) results in cross-area evaluation on CVUSA (Workman et al. 2015), CVACT (Liu and Li, 2019), and VIGOR (Zhu et al. 2021) by a large margin (16.44%, 22.71%, and 13.66% without polar transformation) while keeping the same-area performance comparable to existing SOTA. Moreover, we provide detailed analyses of GeoDTR+.

SAMFL: Secure Aggregation Mechanism for Federated Learning with Byzantine-robustness by functional encryption

Federated learning (FL) enables collaborative model training without sharing private data, thereby potentially meeting the growing demand for data privacy protection. Despite its potentials, FL also poses challenges in achieving privacy-preservation and Byzantine-robustness when handling sensitive data. To address these challenges, we present a novel Secure Aggregation Mechanism for Federated Learning with Byzantine-Robustness by Functional Encryption (SAMFL). Our approach designs a novel dual-decryption multi-input functional encryption (DD-MIFE) scheme, which enables efficient computation of cosine similarities and aggregation of encrypted gradients through a single ciphertext. This innovative scheme allows for dual decryption, producing distinct results based on different keys, while maintaining high efficiency. We further propose TF-Init, integrating DD-MIFE with Truth Discovery (TD) to eliminate the reliance on a root dataset. Additionally, we devise a secure cosine similarity calculation aggregation protocol (SC2AP) using DD-MIFE, ensuring privacy-preserving and Byzantine-robust FL secure aggregation. To enhance FL efficiency, we employ single instruction multiple data (SIMD) to parallelize encryption and decryption processes. Concurrently, to preserve accuracy, we incorporate differential privacy (DP) with selective clipping of model layers within the FL framework. Finally, we integrate TF-Init, SC2AP, SIMD, and DP to construct SAMFL. Extensive experiments demonstrate that SAMFL successfully defends against both inference attacks and poisoning attacks, while improving efficiency and accuracy compared to existing methods. SAMFL provides a comprehensive integrated solution for FL with efficiency, accuracy, privacy-preservation, and robustness.

Simultaneously mapping the 3D distributions of multiple heavy metals in an industrial site using deep learning and multisource auxiliary data

Three-dimensional (3D) distributions of multiple soil pollutants in industrial site are crucial for risk assessment and remediation. Yet, their 3D prediction accuracies are often low because of the strong variability of pollutants and availability of 3D covariate data. This study proposed a patch-based multi-task convolution neural network (MT-CNN) model for simultaneously predicting the 3D distributions of Zn, Pb, Ni, and Cu at an industrial site. By integrating neighborhood patches from multisource covariates, the MT-CNN model captured both horizontal and vertical pollution information, and outperformed the widely-used methods such as random forest (RF), ordinary Kriging (OK), and inverse distance weighting (IDW) for all the 4 heavy metals, with R2 values of 0.58, 0.56, 0.29 and 0.23 for Zn, Pb, Ni and Cu, respectively. Besides, the MT-CNN model achieved more stable predictions with reasonable accuracy, in comparison with the single-task CNN model. These results highlighted the potential of the proposed MT-CNN in simultaneously mapping the 3D distributions of multiple pollutants, while balancing the model training, maintaining and accuracy for low-cost rapid assessment of soil pollution at industrial sites.

Enhancing Amyloid PET Quantification: MRI-Guided Super-Resolution Using Latent Diffusion Models

Amyloid PET imaging plays a crucial role in the diagnosis and research of Alzheimer’s disease (AD), allowing non-invasive detection of amyloid-β plaques in the brain. However, the low spatial resolution of PET scans limits the accurate quantification of amyloid deposition due to partial volume effects (PVE). In this study, we propose a novel approach to addressing PVE using a latent diffusion model for resolution recovery (LDM-RR) of PET imaging. We leverage a synthetic data generation pipeline to create high-resolution PET digital phantoms for model training. The proposed LDM-RR model incorporates a weighted combination of L1, L2, and MS-SSIM losses at both noise and image scales to enhance MRI-guided reconstruction. We evaluated the model’s performance in improving statistical power for detecting longitudinal changes and enhancing agreement between amyloid PET measurements from different tracers. The results demonstrate that the LDM-RR approach significantly improves PET quantification accuracy, reduces inter-tracer variability, and enhances the detection of subtle changes in amyloid deposition over time. We show that deep learning has the potential to improve PET quantification in AD, effectively contributing to the early detection and monitoring of disease progression.

A convolutional neural network to control sound level for air conditioning units in four different classroom conditions

Air conditioning units (ACUs) are widely used in educational areas like classrooms in order to ensure the occupant’s well-being and to control CO2 concentration (by ventilating using fresh outdoor air). However, the noise level of these ACUs can disrupt the learning environment. Consequently, we propose an approach based on machine learning that is able to distinguish between the different acoustic situations in a classroom, and to dynamically adjust the air volume flow accordingly. To this end, the present algorithm was trained with sound recordings in four different scenarios in a classroom. Both Mel spectrogram and Cochleagram are considered and applied for the task of training the convolutional neural networks (CNN) model, thus enhancing published works in the literature, which only considered the Mel spectrogram. Results show how the Cochleagram is pre-eminent to handle the CNN model training over the Mel spectrogram. Accordingly, we use the Cochleagram for the CNN model training, which is subsequently used for detecting the current situation in the room and adapting the ACU operation to this situation. The results show that running ACUs in high mode provides a learning environment with low CO2 concentration and high noise. Instead, controlling ACU based scenario predictions made by the CNN model provides a good learning environment with adequate CO2 concentration and acceptable noise level. Our results demonstrate the effectiveness of using sound classification as a trigger for ventilation control, with the CNN model achieving a good accuracy rate in sound recognition. This underscores the potential of integrating advanced machine learning techniques into building management systems to foster environments that adapt to the needs of their inhabitants automatically. The results of the present work are useful for improving the comfort of the occupants through dynamic ACUs adjustments based on acoustical situations.

A high-precision automatic diagnosis method of maize developmental stage based on ensemble deep learning with IoT devices

Accurately determining the stage of crop development holds significant importance for field crop management. With the advancement of smart agriculture, an increasing number of Internet of Things (IoT) devices are being integrated into agricultural production, enabling more efficient acquisition of high-precision crop images. Currently, research on detecting crop growth stages based on IoT device images remains relatively scarce. Most existing studies rely on a single network model for detection, often encountering issues such as low accuracy and overfitting. Therefore, in this study, we collected maize images using IoT devices and constructed an integrated deep learning model by utilizing four convolutional neural networks (CNNs) to detect the growth period of maize in real time. Additionally, we implemented several improvements on these four CNNs and subsequently tested the performance of the ensemble model on the maize dataset. Regarding the ensemble strategy for the ensemble model, we proposed a dynamic weighted voting method, building upon the original voting approach, which can mitigate model training fluctuations and expedite model convergence. Ultimately, we manually simulated various lighting conditions to assess their impact on the ensemble model. Experimental results demonstrate that the ensemble deep model proposed in this paper represents a robust method for detecting maize growth stages, achieving an accuracy rate of 0.976 on the maize dataset, effectively facilitating high-precision detection of maize growth stages in complex backgrounds.

Diagnosis of Oil-Drilling Permanent Magnet Synchronous Motor Interturn Short Circuit Under Physical Fault Model Training Sample

Diagnosis of Oil-Drilling Permanent Magnet Synchronous Motor Interturn Short Circuit Under Physical Fault Model Training Sample

Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models.

In deep learning, different kinds of deep networks typically need different optimizers, which have to be chosen after multiple trials, making the training process inefficient. To relieve this issue and consistently improve the model training speed across deep networks, we propose the ADAptive Nesterov momentum algorithm, Adan for short. Adan first reformulates the vanilla Nesterov acceleration to develop a new Nesterov momentum estimation (NME) method, which avoids the extra overhead of computing gradient at the extrapolation point. Then Adan adopts NME to estimate the gradient's first- and second-order moments in adaptive gradient algorithms for convergence acceleration. Besides, we prove that Adan finds an ϵ-approximate first-order stationary point within O(ϵ-3.5) stochastic gradient complexity on the non-convex stochastic problems (e.g., deep learning problems), matching the best-known lower bound. Extensive experimental results show that Adan consistently surpasses the corresponding SoTA optimizers on vision, language, and RL tasks and sets new SoTAs for many popular networks and frameworks, e.g., ResNet, ConvNext, ViT, Swin, MAE, DETR, GPT-2, Transformer-XL, and BERT. More surprisingly, Adan can use half of the training cost (epochs) of SoTA optimizers to achieve higher or comparable performance on ViT, GPT-2, MAE, etc, and also shows great tolerance to a large range of minibatch size, e.g., from 1 k to 32 k.

Improved landslide susceptibility assessment: A new negative sample collection strategy and a comparative analysis of zoning methods

Landslide susceptibility assessment (LSA) aims to determine the spatial probability of landslides, reducing the loss caused by future landslides. In order to assess the impact of various negative sample collection strategies on the prediction accuracy of the landslide susceptibility assessment (LSA) model, and to investigate the effectiveness of landslide susceptibility zoning methods. Taking Fengjie County, Chongqing City, China as the study area, this study proposes three negative sample collection strategies based on slope unit, buffer zone, and information value, and combines them with C5.0 decision tree (DT) model respectively to construct an LSA model. Concurrently, the landslide susceptibility indexes (LSIs) were divided using the K-means clustering algorithm and contrasted with the natural breakpoint classification (NBC), quantile classification (QC), equal interval classification (EIC), and geometric interval classification (GIC) methods. The results show that: (1) Rainfall, elevation, and water system are the primary conditioning factors of landslide development in the study area. (2) The accuracy of the negative sample collection strategy based on the slope units on the model training subset and the test subset reached 97.78 % and 92.99 %, respectively, and the AUC values were 0.978 and 0.930, indicating high model accuracy. (3) The zoning effect based on the K-means clustering algorithm was the best, and the predicted very-high and high susceptibility areas were 805.73 km2 and 567.66 km2, respectively, accounting for 19.59 % and 13.80 % of Fengjie County. The very-high and high susceptibility areas had maximum FR values of 3.963 and 1.432, respectively, when compared to other zoning methods. This study can provide a more objective and scientific method for LSA, and the findings can offer more precise decision assistance for risk management and geological disaster prevention.

Impact of federated learning and explainable artificial intelligence for medical image diagnosis

Medical image recognition has enormous potential to benefit from the recent developments in federated learning (FL) and interpretable artificial intelligence (AI). The function of FL and explainable artificial intelligence (XAI) in the diagnosis of brain cancers is discussed in this paper. XAI and FL techniques are vital for ensuring data ethics during medical image processing. This paper highlights the benefits of FL, such as cooperative model training and data privacy preservation, and the significance of XAI approaches in providing logical justifications for model predictions. A number of case studies on the segmentation of medical images employing FL were reviewed to compares and contrasts various methods for assessing the efficacy of FL and XAI based diagnostic models for brain tumors. The relevance of FL and XAI to improve the accuracy and interpretability during medical image diagnosis have been presented. Future research directions are also described indicating as to integrate data from various modes, create standardised evaluation processes, and manage ethical issues. This paper is intended to provide a deeper insight on relevance of FL and XAI in medical image diagnosis.

Embracing Federated Learning: Enabling Weak Client Participation via Partial Model Training

Embracing Federated Learning: Enabling Weak Client Participation via Partial Model Training

PopFL: A scalable Federated Learning model in serverless edge computing integrating with dynamic pop-up network

With the rapid increase in the number of Internet-of-Things (IoT) devices, the massive volume of data creates significant challenges for traditional cloud-based solutions. These solutions often lead to high latency, increased operational costs, and limited scalability, making them unsuitable for real-time applications and resource-constrained environments. As a result, edge, and fog computing have emerged as viable alternatives, reducing latency and costs by processing data closer to its source. However, managing the flow of such vast and distributed data streams requires well-structured data pipelines to control the complete lifecycle—from data acquisition at the source to processing at the edge and fog layers, and finally storage and analytics in the cloud. To dynamically handle data analytics at varying distances from the source, often on heterogeneous hardware devices with collaborative learning techniques such as Federated Learning (FL). FL enables decentralized model training by leveraging the local data on Edge Devices (EDs), thereby preserving data privacy and reducing communication overhead with the cloud. However, FL faces critical challenges, including data heterogeneity, where the non-independent and identically distributed (non-IID) nature of data degrades model performance, and resource limitations on EDs, which lead to inefficiencies in training and biases in the aggregated models.To address these issues, we propose a novel FL solution, called Pop-Up Federated Learning (PopFL) in edge networks. This solution introduces hierarchical aggregation to reduce network congestion by distributing the aggregation tasks across multiple Fog Servers (FSs), rather than relying solely on centralized cloud aggregation. To further enhance participation and resource utilization at the edge, we incorporate the Stackelberg game model, which incentivizes EDs based on their contribution and resource availability. Additionally, PopFL employs a pop-up ad-hoc network for scalable and efficient communication between EDs and FSs, ensuring robust data transmission in dynamic network conditions. Extensive experiments conducted on three diverse datasets highlight the superior performance of PopFL compared to state-of-the-art FL techniques. The results show significant improvements in model accuracy, robustness, and fairness across various scenarios, effectively addressing the challenges of data heterogeneity and resource limitations. Through these innovations, PopFL paves the way for more reliable and efficient distributed learning systems, unlocking the full potential of FL in real-world applications where low latency and scalable solutions are critical.

Sensor attack online classification for UAVs using machine learning

Unmanned Aerial Vehicle (UAV) sensors play a vital role in maintaining flight safety and stability. However, the increasing frequency and complexity of sensor attacks have emerged as a critical threat to UAV systems. The current lack of robust multi-classification methods for detecting sensor attacks limits the effectiveness and completeness of existing defense strategies. This research addresses these challenges by leveraging machine learning (ML) techniques to classify various sensor attacks using heterogeneous sensor data and control parameters, thereby enhancing UAV system security. In this study, we design and implement multiple sensor attack scenarios targeting gyroscopes, accelerometers, barometers, and GPS. Comprehensive datasets are collected during UAV flight, integrating diverse sensor readings, flight states, and control parameters. By analyzing the characteristics of sensor attacks and their impact on position estimation and attitude control, we identify and extract key features. To optimize the classification model, we employ feature importance analysis, correlation analysis, and ablation experiments, significantly reducing data dimensionality and enhancing model training efficiency. The experimental results demonstrate the proposed ML-based multi-classification model’s superior performance, achieving a detection rate of 89.38%, significantly outperforming traditional single-attack detection methods in terms of generalization capability. Our approach efficiently handles complex multi-sensor attack scenarios. Moreover, deploying the optimized model on UAV firmware enables real-time monitoring and classification, achieving an online detection rate of 74% with a response time of approximately 0.495ms per detection. The model’s lightweight design, requiring only 48KB of storage, makes it ideal for resource-constrained UAV environments. These contributions highlight the potential of our approach to enhance real-time anomaly detection and improve UAV system resilience against diverse sensor attacks.

Crossing Domains: The Role of Transfer Learning in Rapid AI Prototyping and Deployment

In the changing world of technology today it's crucial for AI advancements to meet the increasing need for efficiency and top-notch performance. One major hurdle in developing and implementing AI solutions is the requirement, for sets of specialized data and computing power which often slows down progress. Conventional machine learning models usually demand starting from scratch with each project result in significant time and financial investments. Transfer learning is a game changer as it allows the sharing of expertise and pre-existing models from one area to assist with tasks in another domain seamlessly bridging the gap between them This method significantly lessens the reliance on datasets lowers computational requirements and speeds up the overall process of developing AI technologies In this piece we delve into how transfer learning is pivotal in transferring knowledge across domains especially beneficial, for quickly prototyping and deploying AI solutions In our analysis of different uses like natural language processing and medical diagnostics as well as computer vision applications we show how transfer learning boosts effectiveness in scenarios with limited data availability and speeds up the implementation of models while also allowing for better adjustments to pre trained models for particular tasks. Additionally, we explore the potential ahead for transfer learning to support the development of AI models that are easily customized for new tasks, in various fields rapidly. The growing access to trained models and the ongoing progress, in transfer learning methods are setting the stage for AI to be more user friendly and effective – enabling industries to drive innovation and roll out solutions at an unprecedented speed. Keywords: Transfer Learning, AI Prototyping, AI Deployment, Pre-trained Models, Cross-Domain Applications, Machine Learning, Natural Language Processing, Computer Vision, Medical Imaging, Model Fine-tuning, Data Scarcity, Scalability in AI.

Second Version on the Product Color Variation Management using Artificial Intelligence

This research explores using artificial intelligence (AI) for managing color variations in products to boost market performance by optimizing product aesthetics and aligning with consumer preferences. The study investigates a system that leverages AI, deep learning, and neural networks to analyze real-time consumer data, including product preferences, buying history, and sales history. An AI model was created to predict and modify product colors dynamically, aiming to maximize consumer appeal and engagement. The system workflow includes stages for data gathering, processing, feature extraction, model training, software integration, and testing. AI-driven interventions were evaluated through consumer satisfaction, sales metrics, and digital engagement analytics, illustrating the potential for AI to influence product design effectively. This research suggests a transformative best practice for consumer-centered marketing, where AI facilitates color customization aligned with evolving consumer trends. (Sasibhushan Rao Chanthati, 2022)

Random Forest and Decision Tree Facies Classification Models for Well Log Data of the Mishrif Formation from Basrah Oil Company, Southern Iraq

Facies collected from wells drilled in the study area were interpreted manually by using cores at every 10 meters of depth during well drilling. This depth of cores does not give true facies of all wells because the cores every 10 meters are considered very large. Extracting cores is financially expensive and takes a long time. The methodology of machine learning consists of four steps (Data gathering, Data preprocessing, Model training, and Model evaluation). This work intends to apply two of the supervised machine learning techniques random forest and decision tree models. (1) Data gathering, this dataset was collected from the Basrah Oil Company. It contains of ten wells (B-3, B-4, B-5, B-15, B-17, B-18, B-19, B-34, B-39, and B-40). Every well contains six features (logs): Sonic log, Resistivity Deep, Micro Spherically Focused Log, Neutron porosity, Density log, and Gamma-ray. Also, the dataset contains ten facies labels: mudstone, wack stone, packstone, roundstone, floatstone, shale, mud and wack stone, wack and pack stone, pack and grain stone and pack and float stone. These logs cover all the thickness of the Mishrif Formation, which is the goal of our study. (2) Preprocess, the data must be cleaned of outlier values; these values reduce the accuracy of the model during training. It is necessary at this stage to understand the relationships between all the features, because the highly correlated relationship between any two features, the more useless it will be in machine learning. The well B-5 blinded it for training to demonstrate the ability of machine learning models to predict lithofacies, then splitting randomly the dataset into 70% for training, validation 10%, and 20% for testing to verify the performance. (3) Training two models machine learning models decision tree and random forest. (4) Four statistics are computed for two models from the confusion matrix (accuracy of classification, recall, precision, and F1-score) showing the random forest was more accurate than the decision tree because the random forest model deals very well with this amount of dataset. Receiver Operating Characteristics curves of the random forest model have obtained the largest Area Under the Curve than the decision tree, is positive and above the main diagonal for all lithotypes, and the values for all classes reached more than 95%, except class 6 (shale), because of the 100% accuracy of classification. Facies classification by machine learning approach has two benefits (1) Increased accuracy of describing oil reservoirs and (2) Reducing the time, which a geologist needs to interpret logs data.

Detection of Fraud and Malware Apps in Google Play

The proliferation of mobile applications on Google Play has led to an increased risk of fraudulent and malicious apps that can compromise user security and privacy. This study presents a comprehensive approach to detecting fraud and malware in Google Play apps using machine learning (ML) and deep learning (DL) techniques. A Flask-based backend is developed to facilitate the upload, processing, and analysis of datasets containing app features and labels. The system utilizes Support Vector Machine (SVM) and Artificial Neural Network (ANN) models to classify apps as benign or malicious. Key stages include data preprocessing, dataset splitting, model training, and performance evaluation. The models' accuracies are compared, and results are visualized to demonstrate their effectiveness. Additionally, an APK upload feature simulates real-time analysis and prediction, enhancing the practical applicability of the solution. The integration of ML and DL methods provides a robust framework for proactively identifying and mitigating threats posed by fraudulent and malware apps in the Google Play ecosystem.

Credit Risk Modelling for Small and Medium Enterprises (SMEs) Using Machine Learning

This study focuses on developing a credit risk model specifically designed for Small and Medium Enterprises (SMEs) using advanced machine learning techniques. The purpose is to enhance the precision of credit risk assessments, providing financial institutions with reliable tools to gauge the likelihood of default among SMEs. The methodology includes data collection from traditional and alternative sources, followed by feature engineering and model training using algorithms such as logistic regression, decision trees, and neural networks. Results demonstrate significant improvements in predictive accuracy, enabling financial institutions to make data-driven lending decisions and manage risks more effectively. This model not only contributes to financial stability but also promotes increased access to credit for SMEs.

Predictive Analytics for Maintenance

We present the development of a machine learning-based predictive maintenance tool tailored to the industrial industry. Predictive maintenance and operational optimization have turned manufacturing into a manufacturing revolution because to machine learning's capacity to learn from data and generate precise predictions. 96% accuracy was attained in the initial training of a K-means clustering model. An algorithm known as Random Forest was used to increase forecast accuracy, and the outcome was a very good 98% accuracy. Thus, it was decided to apply the Random Forest model. With Flask, a predictive maintenance web interface was created, and the learnt model was easily included to provide real-time predictions. Through dramatically lower equipment downtime and improved operational efficiency, this application highlights the value of machine learning in predictive maintenance in industrial settings.