Adaptive Gradient Research Articles

CAD-PsorNet: deep transfer learning for computer-assisted diagnosis of skin psoriasis

Psoriasis, being a chronic, inflammatory, lifelong skin disorder, has become a major threat to the human population. The precise and effective diagnosis of psoriasis continues to be difficult for clinicians due to its varied nature. In northern India, the prevalence of psoriasis among adult population ranges from 0.44 to 2.8%. Chronic plaque psoriasis accounts for over 90% of cases. This study utilized a dataset of 325 raw images collected from a reputable local hospital using a digital camera under uniform lighting conditions. These images were processed to generate 496 image patches (both diseased and normal), which were then normalized and resized for model training. An automated psoriasis image recognition framework was developed using four state-of-the-art deep transfer learning models: VGG16, VGG19, MobileNetV1, and ResNet-50. The convolutional layers adopted various edge, shape, and color filters to generate the feature map for psoriasis detection. Each pre-trained model was adapted with two dense layers, one dropout layer, and one output layer to classify input images. Among these models, MobileNetV1 achieved the best performance, with 94.84% sensitivity, 89.37% specificity, and 97.24% overall accuracy. Hyper-parameter tuning was performed using grid search to optimize learning rates, batch sizes, and dropout rates. The AdaGrad (Adaptive gradient)) optimizer was chosen for its adaptive learning rate capabilities, facilitating quicker convergence in model performance. Consequently, the methodology’s performance improved to 94.25% sensitivity, 96.42% specificity, and 99.13% overall accuracy. The model’s performance was also compared with non-machine learning-based diagnostic methods, yielding a Dice coefficient of 0.98. However, the model’s effectiveness is dependent upon high-quality input images, as poor image conditions may affect accuracy, and it may not generalize well across diverse demographics or psoriasis variations, highlighting the need for varied training datasets for robustness.

Policy-driven innovations in fraud prevention: Developing an ARFLGB-XGBoost early warning model to mitigate online romance scams in telecommunication networks

To address the escalating online romance scams within telecom fraud, we developed an Adaptive Random Forest Light Gradient Boosting (ARFLGB)-XGBoost early warning system. Our method involves compiling detailed Online Romance Scams (ORS) incident data into a 24-variable dataset, categorized to analyze feature importance with Random Forest and LightGBM models. An innovative adaptive algorithm, the Adaptive Random Forest Light Gradient Boosting, optimizes these features for integration with XGBoost, enhancing early Online romance scams threat detection. Our model showed significant performance improvements over traditional models, with accuracy gains of 3.9%, a 12.5% increase in precision, recall improvement by 5%, an F1 score increase by 5.6%, and a 5.2% increase in Area Under the Curve (AUC). This research highlights the essential role of advanced fraud detection in preserving communication network integrity, contributing to a stable economy and public safety, with implications for policymakers and industry in advancing secure communication infrastructure.

Diffusion-driven Incomplete Multimodal Learning for Air Quality Prediction

Predicting air quality using multimodal data is crucial to comprehensively capture the diverse factors influencing atmospheric conditions. Therefore, this study introduces a multimodal learning framework that integrates outdoor images with traditional ground-based observations to improve the accuracy and reliability of air quality predictions. However, aligning and fusing these heterogeneous data sources pose a formidable challenge, further exacerbated by pervasive data incompleteness issues in practice. In this paper, we propose a novel incomplete multimodal learning approach (iMMAir) to recovery missing data for robust air quality prediction. Specifically, we first design a shallow feature extractor to capture modal-specific features within the latent representation space. Then we develop a conditional diffusion-driven recovery module to mitigate the distribution gap between the recovered and true data. This module further incorporates two conditional constraints of temporal correlation and semantic consistency for effective modal completion. Finally, we reconstruct incomplete modalities and fuse available data using a multimodal transformer network to predict the air quality. To alleviate the modality imbalance problem, we employ an adaptive gradient modulation strategy to adjust the optimization of each modality. Experimental results demonstrate that iMMAir significantly reduces prediction errors, outperforming baseline models by an average of 5.6% and 2.5% in air quality regression and classification tasks. Our source code and data are available at https://github.com/pestasu/IMMAir.

Ensemble LVQ Model for Photovoltaic Line-to-Line Fault Diagnosis Using K-Means Clustering and AdaGrad

Line-to-line (LL) faults are one of the most frequent short-circuit conditions in photovoltaic (PV) arrays which are conventionally detected and cleared by overcurrent protection devices (OCPDs). However, OCPDs are shown to face challenges when detecting LL faults under critical detection conditions, i.e., low mismatch levels and/or high fault impedance values. This occurs due to insufficient fault current, thus leaving the LL faults undetected and leading to power losses and even catastrophic fire hazards. To compensate for OCPD deficiencies, recent studies have proposed modern artificial intelligence (AI)-based methods. However, various limitations can still be witnessed even in AI-based methods, such as (i) most of the models requiring a massive training dataset, (ii) critical fault detection conditions not being taken into consideration, (iii) models not being accurate enough when dealing with critical conditions, etc. To this end, the present paper proposes a learning vector quantization (LVQ)-based ensemble learning model in which three LVQs are individually trained to detect and classify LL faults in PV arrays. The initial LVQ vectors are determined using the k-means clustering method, and the learning rate is optimized by the adaptive gradient (AdaGrad) optimizer. The training and testing datasets are collected according to the PV array’s current–voltage (I–V) characteristic curve, and several features are extracted based on the Canberra and chi-squared distance techniques. The model utilizes a small training dataset, considers various critical detection conditions for LL faults—such as different mismatch levels and fault impedance values—and the final experimental results show that the model achieves an impressive average accuracy of 99.26%.

A fault diagnosis method for bearings and gears in rotating machinery based on data fusion and transfer learning

Abstract Rotating machinery is a crucial component of industrial equipment, and the fault diagnosis of bearings and gears, as vital elements of rotating machinery, is essential since they often fail under harsh working conditions, leading to significant property losses and serious personal safety problems. However, fault data for gears and bearings are often sparse in actual condition, and it is a challenge to ensure the reliability and stability of fault diagnosis results by extracting the features of a single data. To solve the above problems, this paper proposes a fault diagnosis method that combines Transfer Learning and data fusion techniques. Firstly, in this method, two kinds of fault signals are transformed into Gramian Angular Difference Fields and Recurrence Plot. Next, a U-shaped feature fusion dual discriminator generative adversarial network is used to fuse two-dimensional images from multiple sensor data. Its feature fusion module deeply integrates the features of the two images, thereby solving the impact of single data on the reliability and stability of fault diagnosis. Moreover, open-source datasets are used for Transfer Learning training to tackle the small sample problem. Finally, a decision-level information fusion classifier, the Dual-Branch Dempster-Shafer Classifier (DB-DSC), classifies the fused images. This classifier incorporates an improved soft threshold function and D-S evidence theory to achieve adaptive gradient changes and improve the robustness and accuracy of classification results. The experimental results show the effectiveness and stability of the proposed method, and the generated images get high score in several metrics. The average classification accuracy of the classification network reaches 93% and 92.5% on the two datasets, Therefore, the proposed method exhibits strong fault diagnosis capabilities under the small sample conditions of bearings and gears.

Super Learner Algorithm for Carotid Artery Disease Diagnosis: A Machine Learning Approach Leveraging Craniocervical CT Angiography.

This study introduces a machine learning (ML) approach to diagnosing carotid artery diseases, including stenosis, aneurysm, and dissection, by leveraging craniocervical computed tomography angiography (CTA) data. A meticulously curated, balanced dataset of 122 patient cases was used, ensuring reproducibility and data quality, and this is publicly accessible at (insert dataset location). The proposed method integrates a super learner model which combines adaptive boosting, gradient boosting, and random forests algorithms, achieving an accuracy of 90%. To enhance model robustness and generalization, techniques such as k-fold cross-validation, bootstrapping, data augmentation, and the synthetic minority oversampling technique (SMOTE) were applied, expanding the dataset to 1000 instances and significantly improving performance for minority classes like aneurysm and dissection. The results highlight the pivotal role of blood vessel structural analysis in diagnosing carotid artery diseases and demonstrate the superior performance of the super learner model in comparison with state-of-the-art (SOTA) methods in terms of both accuracy and robustness. This manuscript outlines the methodology, compares the results with state-of-the-art approaches, and provides insights for future research directions in applying machine learning to medical diagnostics.

AWDP-FL: An Adaptive Differential Privacy Federated Learning Framework

Data security and user privacy concerns are receiving increasing attention. Federated learning models based on differential privacy offer a distributed machine learning framework that protects data privacy. However, the noise introduced by the differential privacy mechanism may affect the model’s usability, especially when reasonable gradient clipping is absent. Fluctuations in the gradients can lead to issues like gradient explosion, compromising training stability and potentially leaking privacy. Therefore, gradient clipping has become a crucial method for protecting both model performance and data privacy. To balance privacy protection and model performance, we propose the Adaptive Weight-Based Differential Privacy Federated Learning (AWDP-FL) framework, which processes model gradient parameters at the neural network layer level. First, by designing and recording the change trends of two-layer historical gradient sequences, we analyze and predict gradient variations in the current iteration and calculate the corresponding weight values. Then, based on these weights, we perform adaptive gradient clipping for each data point in each training batch, which is followed by gradient momentum updates based on the third moment. Before uploading the parameters, Gaussian noise is added to protect privacy while maintaining model accuracy. Theoretical analysis and experimental results validate the effectiveness of this framework under strong privacy constraints.

RFCSC: Communication efficient reinforcement federated learning with dynamic client selection and adaptive gradient compression

In the field of public safety, high-quality data is often owned by governments, companies, and organizations, making it difficult to train effective models through centralized datasets. This paper leverages federated learning as a mechanism to address data privacy concerns. Within the federated learning framework, the data across different clients is typically Non-IID (non-Independently and Identically Distributed). Furthermore, The complexity of sensitive image recognition tasks in public safety, along with the large number of model parameters, can lead to communication congestion issues in federated learning. To solve these challenges, this paper proposes a Reinforcement Federated Client Selection and Gradient Compression Method(RFCSC). By integrating client data prototypes with accuracy metrics, we dynamically assess the contribution of each client in federated learning. Through an intelligent dynamic incentive mechanism based on reinforcement learning, high-quality client nodes are dynamically selected to participate in federated learning, achieving dynamic adaptive aggregation, reducing the influence of Non-IID data, reducing communication cost, enhancing model accuracy, and achieving a balance between quality and efficiency. To address the issue of high communication cost for parameter transmission, we propose an adaptive model compression strategy in reinforcement federated learning which make local clients find an appropriate compression rate. This approach not only reduces gradient communication overhead but also minimizes the impact of compression on model accuracy. The effectiveness of our proposed approach are corroborated through comprehensive experiments conducted on one private dataset and two public datasets.

A hierarchical adaptive federated reinforcement learning for efficient resource allocation and task scheduling in hierarchical IoT network

The increasing demand for processing numerous data from IoT devices in a hierarchical IoT network drives researchers to propose different resource allocation methods in the edge hosts efficiently. Traditional approaches often compromise on one of these aspects: either prioritizing local decision-making at the edge, which lacks global system insights or centralizing decisions in cloud systems, which raises privacy concerns. Additionally, most solutions do not consider scheduling tasks at the same time to effectively complete the prioritized task accordingly. This study introduces the hierarchical adaptive federated reinforcement learning (HAFedRL) framework for robust resource allocation and task scheduling in hierarchical IoT networks. At the local edge host level, a primal–dual update based deep deterministic policy gradient (DDPG) method is introduced for effective individual task resource allocation and scheduling. Concurrently, the central server utilizes an adaptive multi-objective policy gradient (AMOPG) which integrates a multi-objective policy adaptation (MOPA) with dynamic federated reward aggregation (DFRA) method to allocate resources across connected edge hosts. An adaptive learning rate modulation (ALRM) is proposed for faster convergence and to ensure high performance output from HAFedRL. Our proposed HAFedRL enables the effective integration of reward from edge hosts, ensuring the alignment of local and global optimization goals. The experimental results of HAFedRL showcase its efficacy in improving system-wide utility, average task completion rate, and optimizing resource utilization, establishing it as a robust solution for hierarchical IoT networks.

Adaptive sampling and reconstruction for gradient-domain rendering

Gradient-domain rendering estimates finite difference gradients of image intensities and reconstructs the final result by solving a screened Poisson problem, which shows improvements over merely sampling pixel intensities. Adaptive sampling is another orthogonal research area that focuses on distributing samples adaptively in the primal domain. However, adaptive sampling in the gradient domain with low sampling budget has been less explored. Our idea is based on the observation that signals in the gradient domain are sparse, which provides more flexibility for adaptive sampling. We propose a deep-learning-based end-to-end sampling and reconstruction framework in gradient-domain rendering, enabling adaptive sampling gradient and the primal maps simultaneously. We conducted extensive experiments for evaluation and showed that our method produces better reconstruction quality than other methods in the test dataset.

A radioactive source-seeking method based on angle constraint and particle diffusion

The deployment of mobile robots to conduct search for radioactive sources plays a crucial role in preventing radiation pollution and safeguarding public health. And it is a significant challenge to accurately locate radioactive sources in unknown environments. Herein, a particle filter based on mobile robot search method is proposed to localize radiation sources. By applying an angle constraint technique, the complexity of this algorithm is reduced, the high accuracy of source estimation is maintained, and the range of search is expanded. Additionally, a particle diffusion technology is applied to address particle loss that occurs during filtering. Furthermore, an adaptive gradient descent strategy is adopted to enhance the search efficiency. Experimental results demonstrate that this method achieves a success rate of over 95 % within a rectangular area (e.g., 40m by 40m), outperforming traditional methods, and it could perform effectively in dual radioactive source search tasks.

Multi-Dimensional Adaptive Learning Rate Gradient Descent Optimization Algorithm for Network Training in Magneto-Optical Defect Detection

Multi-Dimensional Adaptive Learning Rate Gradient Descent Optimization Algorithm for Network Training in Magneto-Optical Defect Detection

AGS: Transferable adversarial attack for person re-identification by adaptive gradient similarity attack

Person re-identification (Re-ID) has achieved tremendous success in the fields of computer vision and security. However, Re-ID models are susceptible to adversarial examples, which are crafted by introducing imperceptible perturbations to benign person images. These adversarial examples often display high success rates in white-box settings but their transferability to black-box settings is relatively low. To improve the transferability of adversarial examples, this paper proposes a novel approach called the adaptive gradient similarity attack (AGS), which encompasses two essential components: gradient similarity and enhanced second moment. Specifically, a gradient similarity modulation is established to better harness the information in the neighborhood of the adjacent input, which can adaptively correct the update direction. Additionally, this paper formulates an enhanced second moment to adjust the update step at each iteration to address the issue of poor transferability. Extensive experiments confirm that AGS achieves the best performance compared with the state-of-the-art gradient-based attacks. Moreover, AGS is a versatile approach that can be integrated with existing input transformation attack techniques. Code is available at https://github.com/ZezeTao/similar_Attack4.

Deep learning-based prediction of nodal metastasis in lung cancer using endobronchial ultrasound

ObjectiveEndobronchial ultrasound-guided transbronchial needle aspiration is a vital tool for mediastinal and hilar lymph node staging in patients with lung cancer. Despite its high diagnostic performance and safety, it has a limited negative predictive value. Our objective was to evaluate the diagnostic performance of deep learning-based prediction of lung cancer lymph node metastases using convolutional neural networks developed from automatically extracted images of endobronchial ultrasound videos without supervision of the lymph node location. MethodsPatient and lymph node data were collected from a single-center database. The diagnosis of metastasis was confirmed with endobronchial ultrasound-guided transbronchial needle aspiration and/or surgically resected specimens; the diagnosis of normal lymph node was confirmed with surgically resected specimens only. An annotation system facilitated automated image extraction from endobronchial ultrasound videos. Image frames were randomly selected and split into training and validation datasets on a per-patient basis. A deep learning model with convolutional neural networks, SqueezeNet, was used for image classification via transfer learning based on pretraining from ImageNet. Adaptive moment estimation and stochastic gradient descent were applied as optimizers. ResultsSqueezeNet, with adaptive moment estimation, achieved a sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of 96.7% each after 300 epochs, whereas SqueezeNet with stochastic gradient descent achieved 91.1% each. However, SqueezeNet with stochastic gradient descent demonstrated more stable performance than with adaptive moment estimation. ConclusionsDeep learning-based image classification using convolutional neural networks showed promising diagnostic accuracy for lung cancer nodal metastasis. Future clinical trials are warranted to validate the algorithm's efficacy in a prospective, large-cohort study.

The Risk of Virus Emergence in South America: A Subtle Balance Between Increasingly Favorable Conditions and a Protective Environment.

South American ecosystems host astonishing biodiversity, with potentially great richness in viruses. However, these ecosystems have not yet been the source of any widespread, epidemic viruses. Here we explore a set of putative causes that may explain this apparent paradox. We discuss that human presence in South America is recent, beginning around 14,000years ago; that few domestications of native species have occurred; and that successive immigration events associated with Old World virus introductions reduced the likelihood of spillovers and adaptation of local viruses into humans. Also, the diversity and ecological characteristics of vertebrate hosts might serve as protective factors. Moreover, although forest areas remained well preserved until recently, current brutal, sudden, and large-scale clear cuts through the forest have resulted in nearly no ecotones, which are essential for creating an adaptive gradient of microbes, hosts, and vectors. This may be temporarily preventing virus emergence. Nevertheless, the mid-term effect of such drastic changes in habitats and landscapes, coupled with explosive urbanization and climate changes, must not be overlooked by health authorities.

Distributed Adaptive Gradient Algorithm With Gradient Tracking for Stochastic Nonconvex Optimization

Distributed Adaptive Gradient Algorithm With Gradient Tracking for Stochastic Nonconvex Optimization

Hybrid convolutional neural network optimized with an artificial algae algorithm for glaucoma screening using fundus images.

We developed an optimized decision support system for retinal fundus image-based glaucoma screening. We combined computer vision algorithms with a convolutional network for fundus images and applied a faster region-based convolutional neural network (FRCNN) and artificial algae algorithm with support vector machine (AAASVM) classifiers. Optic boundary detection, optic cup, and optic disc segmentations were conducted using TernausNet. Glaucoma screening was performed using the optimized FRCNN. The Softmax layer was replaced with an SVM classifier layer and optimized with an AAA to attain enhanced accuracy. Using three retinal fundus image datasets (G1020, digital retinal images vessel extraction, and high-resolution fundus), we obtained accuracy of 95.11%, 92.87%, and 93.7%, respectively. Framework accuracy was amplified with an adaptive gradient algorithm optimizer FRCNN (AFRCNN), which achieved average accuracy 94.06%, sensitivity 93.353%, and specificity 94.706%. AAASVM obtained average accuracy of 96.52%, which was 3% ahead of the FRCNN classifier. These classifiers had areas under the curve of 0.9, 0.85, and 0.87, respectively. Based on statistical Friedman evaluation, AAASVM was the best glaucoma screening model. Segmented and classified images can be directed to the health care system to assess patients' progress. This computer-aided decision support system will be useful for optometrists.

A novel optical orthogonality control approach based on noise propagation and its application in spin-exchange relaxation-free co-magnetometers

This paper presents a non-destructive optical orthogonality control method based on adaptive gradient descent for spin-exchange relaxation-free co-magnetometers (SERFCM). The spin-exchange optical pumping technique serves as the foundation of SERFCM measurement. Analyzing the impact of pump power fluctuation on the transverse polarizability of alkali metal atoms in the presence of non-orthogonal beams, considering stochastic noise propagation, is addressed. Subsequently, an optical orthogonal calibration scheme is proposed. Leveraging the proposed orthogonal calibration scheme, an adaptive gradient descent method based on the reference model is formulated, and simulation validates its effectiveness in addressing the rapid online optimization control of SERFCM optical quadrature. Experimental results indicate that the proposed optical orthogonal calibration scheme offers higher accuracy, with the proposed optical orthogonal control algorithm saving 34% of the time compared to the traditional algorithm. Finally, the stability of SERFCM is evaluated using Allan variance, demonstrating that the adjusted SERFCM enhances short-term stability by 84% at around 1s and long-term stability by 639% at around 300s.

EVFL: Towards Efficient Verifiable Federated Learning via Parameter Reuse and Adaptive Sparsification

Federated learning (FL) demonstrates significant potential in Industrial Internet of Things (IIoT) settings, as it allows multiple institutions to jointly construct a shared learning model by exchanging model parameters or gradient updates without the need to transmit raw data. However, FL faces risks related to data poisoning and model poisoning. To address these issues, we propose an efficient verifiable federated learning (EVFL) method, which integrates adaptive gradient sparsification (AdaGS), Boneh–Lynn–Shacham (BLS) signatures, and fully homomorphic encryption (FHE). The combination of BLS signatures and the AdaGS algorithm is used to build a secure aggregation protocol. These protocols verify the integrity of parameters uploaded by industrial agents and the consistency of the server’s aggregation results. Simulation experiments demonstrate that the AdaGS algorithm significantly reduces verification overhead through parameter sparsification and reuse. Our proposed algorithm achieves better verification efficiency compared to existing solutions.

A Gastrointestinal Image Classification Method Based on Improved Adam Algorithm

In this study, a gastrointestinal image classification method based on the improved Adam algorithm is proposed. Gastrointestinal image classification is of great significance in the field of medical image analysis, but it presents numerous challenges, including slow convergence, susceptibility to local minima, and the complexity and imbalance of medical image data. Although the Adam algorithm is widely used in stochastic gradient descent, it tends to suffer from overfitting and gradient explosion issues when dealing with complex data. To address these problems, this paper proposes an improved Adam algorithm, AdamW_AGC, which combines the weight decay and Adaptive Gradient Clipping (AGC) strategies. Weight decay is a common regularization technique used to prevent machine learning models from overfitting. Adaptive gradient clipping avoids the gradient explosion problem by restricting the gradient to a suitable range and helps accelerate the convergence of the optimization process. In order to verify the effectiveness of the proposed algorithm, we conducted experiments on the HyperKvasir dataset and validation experiments on the MNIST and CIFAR10 standard datasets. Experimental results on the HyperKvasir dataset demonstrate that the improved algorithm achieved a classification accuracy of 75.8%, compared to 74.2% for the traditional Adam algorithm, representing an improvement of 1.6%. Furthermore, validation experiments on the MNIST and CIFAR10 datasets resulted in classification accuracies of 98.69% and 71.7%, respectively. These results indicate that the AdamW_AGC algorithm has advantages in handling complex, high-dimensional medical image classification tasks, effectively improving both classification accuracy and training stability. This study provides new ideas and expansions for future optimizer research.