Rate Adaptation Research Articles

A deep-learning-based MAC for integrating channel access, rate adaptation, and channel switch

With increasing density and heterogeneity in unlicensed wireless networks, traditional MAC protocols, such as Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) in Wi-Fi networks, are experiencing performance degradation. This is manifested in increased collisions and extended backoff times, leading to diminished spectrum efficiency and protocol coordination. Addressing these issues, this paper proposes a deep-learning-based MAC paradigm, dubbed DL-MAC, which leverages spectrum data readily available from energy detection modules in wireless devices to achieve the MAC functionalities of channel access, rate adaptation, and channel switch. First, we utilize DL-MAC to realize a joint design of channel access and rate adaptation. Subsequently, we integrate the capability of channel switching into DL-MAC, enhancing its functionality from single-channel to multi-channel operations. Specifically, the DL-MAC protocol incorporates a Deep Neural Network (DNN) for channel selection and a Recurrent Neural Network (RNN) for the joint design of channel access and rate adaptation. We conducted real-world data collection within the 2.4 GHz frequency band to validate the effectiveness of DL-MAC. Experimental results demonstrate that DL-MAC exhibits significantly superior performance compared to traditional algorithms in both single and multi-channel environments, and also outperforms single-function designs. Additionally, the performance of DL-MAC remains robust, unaffected by channel switch overheads within the evaluation range.

A hierarchical adaptive federated reinforcement learning for efficient resource allocation and task scheduling in hierarchical IoT network

The increasing demand for processing numerous data from IoT devices in a hierarchical IoT network drives researchers to propose different resource allocation methods in the edge hosts efficiently. Traditional approaches often compromise on one of these aspects: either prioritizing local decision-making at the edge, which lacks global system insights or centralizing decisions in cloud systems, which raises privacy concerns. Additionally, most solutions do not consider scheduling tasks at the same time to effectively complete the prioritized task accordingly. This study introduces the hierarchical adaptive federated reinforcement learning (HAFedRL) framework for robust resource allocation and task scheduling in hierarchical IoT networks. At the local edge host level, a primal–dual update based deep deterministic policy gradient (DDPG) method is introduced for effective individual task resource allocation and scheduling. Concurrently, the central server utilizes an adaptive multi-objective policy gradient (AMOPG) which integrates a multi-objective policy adaptation (MOPA) with dynamic federated reward aggregation (DFRA) method to allocate resources across connected edge hosts. An adaptive learning rate modulation (ALRM) is proposed for faster convergence and to ensure high performance output from HAFedRL. Our proposed HAFedRL enables the effective integration of reward from edge hosts, ensuring the alignment of local and global optimization goals. The experimental results of HAFedRL showcase its efficacy in improving system-wide utility, average task completion rate, and optimizing resource utilization, establishing it as a robust solution for hierarchical IoT networks.

Speed and accuracy investigations of neural network algorithms for ionospheric modelling at an equatorial region

Neural networks are very efficient tools for modeling, including ionospheric modeling. The training algorithm is important for achieving the optimum performance of the trained network. This research is therefore meant to evaluate and compare the performances of ten neural network training algorithms based on their prediction accuracies, and the duration/times taken by each of the algorithms to establish the optimum result. The neural networks were trained using electron density measurements by Radio Occultation (RO) technique from the Constellation Observing System for Meteorology, Ionosphere, and Climate (COSMIC) satellites. Data for the period 2006 through 2021 was used. The networks were trained using about 2.9 million data points collected from the Kano region, Nigeria (5-degree rectangular region around geographic: 12.00° N, 8.59° E) after performing data quality control. The training algorithms considered in the work include: Bayesian Regularization (BR); Broyden-Fletcher-Goldfarb-Shanno Quasi-Newton (BFG); Conjugate Gradient with Powell/Beale (CGB); Fletcher-Reeves Conjugate Gradient (CGF); Gradient descent with momentum and adaptive learning rate (GDX); Levenberg Marquardt (LM); One Step Secant (OSS); Polak-Ribiére Conjugate Gradient (CGP); Resilient Backpropagation (RP); and Scaled Conjugate Gradient (SCG). The results showed that the BR and the LM algorithms gave the best performances in minimizing the errors of prediction (the mean RMSEs are respectively 112 and 114 ×103 electrons/cm3), but the RP algorithm, which came third in terms of accuracy, was significantly faster than both the LM and BR algorithms. The worst-performing algorithm in terms of accuracy was the GDX algorithm, although it was the fastest algorithm. The BFG algorithm was the worst-performing algorithm in terms of a combination of speed and accuracy. The developed neural network model was validated using ionosonde electron density measurements obtained from Ilorin, Nigeria (geographic: 8.5° N, 4.5° E; geomagnetic: 1.8° S). A comparison of the neural network, the NeQuick, and the IRI model predictions relative to the ionosonde measurements indicate that the neural network model was the best-performing model; the NN model predictions minimized the mean absolute errors (MAEs) in ∼44% of 399 ionosonde profiles investigated, the IRI model did so in ∼32%, and the NeQuick did so in ∼24%. The MAEs of the NeQuick however exhibited the best (least) variance. In overall, the NN model gave the least (best) mean of the MAEs (∼73 × 103 cm−3), compared to ∼82 × 103 cm−3 given by both the NeQuick and the IRI models, further supporting the idea that neural networks are excellent for present-day ionospheric modeling.

Brain Tumor Segmentation from Optimal MRI Slices Using a Lightweight U-Net

The timely detection and accurate localization of brain tumors is crucial in preserving people’s quality of life. Thankfully, intelligent computational systems have proven invaluable in addressing these challenges. In particular, the UNET model can extract essential pixel-level features to automatically identify the tumor’s location. However, known deep learning-based works usually directly feed the 3D volume into the model, which causes excessive computational complexity. This paper presents an approach to boost the UNET network, reducing computational workload while maintaining superior efficiency in locating brain tumors. This concept could benefit portable or embedded recognition systems with limited resources for operating in real time. This enhancement involves an automatic slice selection from the MRI T2 modality volumetric images containing the most relevant tumor information and implementing an adaptive learning rate to avoid local minima. Compared with the original model (7.7 M parameters), the proposed UNET model uses only 2 M parameters and was tested on the BraTS 2017, 2020, and 2021 datasets. Notably, the BraTS2021 dataset provided outstanding binary metric results: 0.7807 for the Intersection Over the Union (IoU), 0.860 for the Dice Similarity Coefficient (DSC), 0.656 for the Sensitivity, and 0.9964 for the Specificity compared to vanilla UNET.

Enhanced Hybrid Congestion Mitigation Strategy for ‘6LoWPAN-RPL based patient-centric IoHT’

The Internet of Healthcare Things (IoHT) is rapidly evolving, providing new opportunities to enhance healthcare delivery. However, the resource limitation of connected medical sensing devices leads to congestion, resulting in reduced network performance, delayed data transmission, and loss of critical medical information, which can have significant consequences in healthcare. To address this issue, this paper proposes an Enhanced Hybrid Congestion Mitigation Strategy (EHCMS) for IPv6 over low-power wireless personal area networks (6LoWPAN) and routing protocol for low-Power and lossy networks (RPL) based patient-centric IoHT (PC-IoHT). The EHCMS combines several techniques, including traffic management, network topology optimization, and load balancing, to enhance network performance and reduce congestion. The proposed framework is a hybrid strategy that utilizes resource- and traffic-control mechanisms to alleviate congestion in the 6LoWPAN-RPL-based patient-centric IoHT network. For the resource-control-based approach, a congestion-aware composite objective function is designed using a few congestion-specific routing metrics and formulated as a multi-attribute decision-making problem solved using Grey relational analysis (GRA). In addition, a non-linear multi-criteria optimization problem-based transmission rate adaptation mechanism is contrived as a traffic-control scheme for congestion mitigation. The effectiveness of the proposed EHCMS is evaluated using simulations on the Cooja simulator in the Contiki-3.0 OS and compared with existing congestion-alleviating strategies. The results demonstrate that the proposed framework can significantly reduce congestion in the IoHT network, perform better than existing works, and improve the quality of service. This research paper contributes to the field of IoHT by proposing an effective congestion mitigation strategy that enhances the reliability and performance of the PC-IoHT network, ultimately improving the quality of healthcare delivery.

CMA-ES with Learning Rate Adaptation

The covariance matrix adaptation evolution strategy (CMA-ES) is one of the most successful methods for solving continuous black-box optimization problems. A practically useful aspect of CMA-ES is that it can be used without hyperparameter tuning. However, the hyperparameter settings still have a considerable impact on performance, especially for difficult tasks, such as solving multimodal or noisy problems. This study comprehensively explores the impact of learning rate on CMA-ES performance and demonstrates the necessity of a small learning rate by considering ordinary differential equations. Thereafter, it discusses the setting of an ideal learning rate. Based on these discussions, we develop a novel learning rate adaptation mechanism for CMA-ES that maintains a constant signal-to-noise ratio. Additionally, we investigate the behavior of CMA-ES with the proposed learning rate adaptation mechanism through numerical experiments and compare the results with those obtained for CMA-ES with a fixed learning rate and with population size adaptation. The results show that CMA-ES with the proposed learning rate adaptation works well for multimodal and/or noisy problems without extremely expensive learning rate tuning.

DRESS syndrome and tuberculosis: Implementation of a desensitization and re-desensitization protocol to recover antituberculosis drugs in a case series at a specialized TB Unit in Lima, Peru.

Antituberculosis drugs (ATDs) could cause severe and rare reactions, such as Drug Reaction with Eosinophilia and Systemic Symptoms (DRESS) syndrome. Recovering ATDs might guarantee a higher cure rate for tuberculosis patients. Our aim was to evaluate the results of desensitization and re-desensitization to recover ATDs in a case series of patients with DRESS syndrome. A retrospective case series study was conducted on patients with DRESS syndrome due to therapy with ATDs from 2021 to 2023. Desensitization and re-desensitization protocols, designed with an algorithm proposed by the Tuberculosis Specialized Unit of the Dos de Mayo National Hospital in Lima, Peru, were implemented. A total of 18 patients underwent desensitization or re-desensitization protocols, achieving an overall success rate of 72.2%. The average time for the development of DRESS syndrome due to ATDs was 19 days. Rifampicin (84.2%), isoniazid (68.4%), and pyrazinamide (26.3%) were identified as the main drugs responsible for this adverse reaction. All patients presented with fever and skin rash, with an average eosinophil percentage of 16.7% (interquartile range: 4.5-28.8). Organ involvement (liver, kidney, and heart) was observed in 8 patients, but only 2 patients experienced severe complications due to DRESS syndrome. A significant association was found between the number of ATDs used and eosinophil levels (P =.03). The study introduced a desensitization and re-desensitization algorithm for the treatment of DRESS syndrome, notable for its safety, adaptability, and high success rate. This advancement provided healthcare professionals with safer and more effective therapeutic approaches for managing this complex condition.

DISTRIBUTED HIGH-PERFORMANCE COMPUTING METHODS FOR ACCELERATING DEEP LEARNING TRAINING

This paper comprehensively analyzes distributed high-performance computing methods for accelerating deep learning training. We explore the evolution of distributed computing architectures, including data parallelism, model parallelism, and pipeline parallelism, and their hybrid implementations. The study delves into optimization techniques crucial for large-scale training, such as distributed optimization algorithms, gradient compression, and adaptive learning rate methods. We investigate communication-efficient algorithms, including Ring All Reduce variants and decentralized training approaches, which address the scalability challenges in distributed systems. The research examines hardware acceleration and specialized systems, focusing on GPU clusters, custom AI accelerators, high-performance interconnects, and distributed storage systems optimized for deep learning workloads. Finally, we discuss this field's challenges and future directions, including scalability-efficiency trade-offs, fault tolerance, energy efficiency in large-scale training, and emerging trends like federated learning and neuromorphic computing. Our findings highlight the synergy between advanced algorithms, specialized hardware, and optimized system designs in pushing the boundaries of large-scale deep learning, paving the way for future breakthroughs in artificial intelligence.

Multi-Dimensional Adaptive Learning Rate Gradient Descent Optimization Algorithm for Network Training in Magneto-Optical Defect Detection

Multi-Dimensional Adaptive Learning Rate Gradient Descent Optimization Algorithm for Network Training in Magneto-Optical Defect Detection

Lossy Image Compression with Stochastic Quantization

Introduction. Lossy image compression algorithms play a crucial role in various domains, including graphics, and image processing. As image information density increases, so do the resources required for processing and transmission. One of the most prominent approaches to address this challenge is color quantization, proposed by Orchard et al. (1991). This technique optimally maps each pixel of an image to a color from a limited palette, maintaining image resolution while significantly reducing information content. Color quantization can be interpreted as a clustering problem (Krishna et al. (1997), Wan (2019)), where image pixels are represented in a three-dimensional space, with each axis corresponding to the intensity of an RGB channel. The purpose of the paper. Scaling of traditional algorithms like K-Means can be challenging for large data, such as modern images with millions of colors. This paper reframes color quantization as a three-dimensional stochastic transportation problem between the set of image pixels and an optimal color palette, where the number of colors is a predefined hyperparameter. We employ Stochastic Quantization (SQ) with a seeding technique proposed by Arthur et al. (2007) to enhance the scalability of color quantization. This method introduces a probabilistic element to the quantization process, potentially improving efficiency and adaptability to diverse image characteristics. Results. To demonstrate the efficiency of our approach, we present experimental results using images from the ImageNet dataset. These experiments illustrate the performance of our Stochastic Quantization method in terms of compression quality, computational efficiency, and scalability compared to traditional color quantization techniques. Conclusions. This study introduces a scalable algorithm for solving the color quantization problem without memory constraints, demonstrating its efficiency on a subset of images from the ImageNet dataset. The convergence speed of the algorithm can be further enhanced by modifying the update rule with alternative methods to Stochastic Gradient Descent (SGD) that incorporate adaptive learning rates. Moreover, the stochastic nature of the proposed solution enables the utilization of parallelization techniques to simultaneously update the positions of multiple quants, potentially leading to significant performance improvements. This aspect of parallelization and its impact on algorithm efficiency presents a topic for future research. The proposed method not only addresses the limitations of existing color quantization techniques but also opens up new possibilities for optimizing image compression algorithms in resource-constrained environments. Keywords: non-convex optimization, stochastic optimization, stochastic quantization, color quantization, lossy compression.

Specified time dual-group synchronization of uncertain complex chaotic systems

Aiming at the specified time dual-group synchronization problem of multi-wing complex chaotic systems containing uncertain terms and external disturbances, a new specified-time sliding mode control scheme is proposed, which directly synchronizes the complex chaotic system without separating the real and imaginary parts of the complex chaotic system. First, a new specified time stability criterion is used to construct the integral sliding mode surface of the synchronous error system to ensure stable sliding motion within the specified time. Subsequently, a proximity controller is designed to drive the error system to reach and remain on the sliding surface within another specified time, thereby achieving specified-time synchronization. In order to realize the proposed stability concept, this paper introduces a new sliding surface and defines the corresponding control law and adaptive rate. The effectiveness of this scheme is proved through Lyapunov stability theory and specified time stability theory. Numerical simulation results show that the scheme has strong robustness to uncertainties and external disturbances, and the controller is not affected by internal uncertainties and external disturbances. Compared to other stabilization time control schemes, this scheme has a shorter synchronization time. In general, this study introduces complex variables and adopts a scheme in which sliding mode surface parameters and controller parameters can be preset to simultaneously achieve dual-group synchronization of two groups of complex chaotic systems within the complex domain. This study offers greater flexibility, presenting novel ideas and approaches for the synchronization control of complex systems. It holds significant theoretical and practical value, providing valuable references and insights for research and applications in related fields.

Effects of periodic bottlenecks on the dynamics of adaptive evolution in microbial populations.

Population bottlenecks can impact the rate of adaptation in evolving populations. On the one hand, each bottleneck reduces the genetic variation that fuels adaptation. On the other hand, each founder that survives a bottleneck can undergo more generations and leave more descendants in a resource-limited environment, which allows surviving beneficial mutations to spread more quickly. A theoretical model predicted that the rate of fitness gains should be maximized using ~8-fold dilutions. Here we investigate the impact of repeated bottlenecks on the dynamics of adaptation using numerical simulations and experimental populations of Escherichia coli. Our simulations confirm the model's prediction when populations evolve in a regime where beneficial mutations are rare and waiting times between successful mutations are long. However, more extreme dilutions maximize fitness gains in simulations when beneficial mutations are common and clonal interference prevents most of them from fixing. To examine these predictions, we propagated 48 E. coli populations with 2-, 8-, 100-, and 1000-fold dilutions for 150 days. Adaptation began earlier and fitness gains were greater with 100- and 1000-fold dilutions than with 8-fold dilutions, consistent with the simulations when beneficial mutations are common. However, the selection pressures in the 2-fold treatment were qualitatively different from the other treatments, violating a critical assumption of the model and simulations. Thus, varying the dilution factor during periodic bottlenecks can have multiple effects on the dynamics of adaptation caused by differential losses of diversity, different numbers of generations, and altered selection.

BUFIT: FINE-GRAINED DYNAMIC BURST FAULT INJECTION TOOL FOR EMBEDDED field programmable gate array TESTING

Fault injection (FI) is a well-known method to attack embedded systems, particularly advanced FPGAs and microcontrollers physically. The FPGA-based embedded system constitutes SRAM for configuration data storage. Multiple-bit upset is a main threat for FPGAs due to technology scaling and complex application bit files. Space environments additionally incur radiation threats to these devices. This paper proposes burst error modeling and a burst fault injection tool (BUFIT) to address these issues. BUFIT has been proposed with fine-grained and coarse-grained circuits. Built-in instrumented FPGA-based FI is proposed for effectively injecting MBUs in configuration memory with space adaptive rate for accurately estimating soft errors. Evaluation of proposed BUFIT on Kintex-7 target FPGA to various OR 1200-based workloads is given to analyze the speed up of the proposed technique. Results on the OR 1200 processor show that BUFIT is three and two orders of magnitude faster than existing DPR and SCFIT techniques. It uses only 0.4 % CLB overhead and has negligible impact on FFs of target SFPGAs.

Individuals with anxiety and depression use atypical decision strategies in an uncertain world.

Previous studies on reinforcement learning have identified three prominent phenomena: (1) individuals with anxiety or depression exhibit a reduced learning rate compared to healthy subjects; (2) learning rates may increase or decrease in environments with rapidly changing (i.e. volatile) or stable feedback conditions, a phenomenon termed learning rate adaptation; and (3) reduced learning rate adaptation is associated with several psychiatric disorders. In other words, multiple learning rate parameters are needed to account for behavioral differences across participant populations and volatility contexts in this flexible learning rate (FLR) model. Here, we propose an alternative explanation, suggesting that behavioral variation across participant populations and volatile contexts arises from the use of mixed decision strategies. To test this hypothesis, we constructed a mixture-of-strategies (MOS) model and used it to analyze the behaviors of 54 healthy controls and 32 patients with anxiety and depression in volatile reversal learning tasks. Compared to the FLR model, the MOS model can reproduce the three classic phenomena by using a single set of strategy preference parameters without introducing any learning rate differences. In addition, the MOS model can successfully account for several novel behavioral patterns that cannot be explained by the FLR model. Preferences for different strategies also predict individual variations in symptom severity. These findings underscore the importance of considering mixed strategy use in human learning and decision-making and suggest atypical strategy preference as a potential mechanism for learning deficits in psychiatric disorders.

Individuals with anxiety and depression use atypical decision strategies in an uncertain world

Previous studies on reinforcement learning have identified three prominent phenomena: (1) individuals with anxiety or depression exhibit a reduced learning rate compared to healthy subjects; (2) learning rates may increase or decrease in environments with rapidly changing (i.e. volatile) or stable feedback conditions, a phenomenon termed learning rate adaptation; and (3) reduced learning rate adaptation is associated with several psychiatric disorders. In other words, multiple learning rate parameters are needed to account for behavioral differences across participant populations and volatility contexts in this flexible learning rate (FLR) model. Here, we propose an alternative explanation, suggesting that behavioral variation across participant populations and volatile contexts arises from the use of mixed decision strategies. To test this hypothesis, we constructed a mixture-of-strategies (MOS) model and used it to analyze the behaviors of 54 healthy controls and 32 patients with anxiety and depression in volatile reversal learning tasks. Compared to the FLR model, the MOS model can reproduce the three classic phenomena by using a single set of strategy preference parameters without introducing any learning rate differences. In addition, the MOS model can successfully account for several novel behavioral patterns that cannot be explained by the FLR model. Preferences for different strategies also predict individual variations in symptom severity. These findings underscore the importance of considering mixed strategy use in human learning and decision-making and suggest atypical strategy preference as a potential mechanism for learning deficits in psychiatric disorders.

Attention defines the context for implicit sensorimotor adaptation.

Movement errors are used to continuously recalibrate the sensorimotor map, a process known as sensorimotor adaptation. Here we examined how attention influences this automatic and obligatory learning process. Focusing first on spatial attention, we compared conditions in which the visual feedback that provided information about the movement outcome was either attended or unattended. Surprisingly, this manipulation had no effect on the rate of adaptation. We next used a dual-task methodology to examine the influence of attentional resources on adaptation. Here, again, we found no effect of attention, with the rate of adaptation similar under focused or divided attention conditions. Interestingly, we found that attention modulates adaptation in an indirect manner: Attended stimuli serve as cues that define the context for learning. The rate of adaptation was significantly attenuated when the attended stimulus changed from the end of one trial to the start of the next trial. In contrast, similar changes to unattended stimuli had no impact on adaptation. Together, these results suggest that visual attention defines the cues that establish the context for sensorimotor learning.

Exploring the relationship between child temperament, maternal psychiatric symptoms, family environment and infant feeding.

Infant temperament is assumed to be primarily innate. However, newer research suggests that maternal affection impacts ratings of temperament and environmental factors, including feeding method, can also influence infant temperament. This study investigates child temperament and its relationships with maternal psychiatric symptoms, environmental variables and feeding method longitudinally in a cohort of children followed from 6 to 72 months. Differences in temperament by feeding group are also investigated. We hypothesized that maternal psychiatric symptoms, environmental stressors, and impaired family dynamics would have negative impact on child temperament, whereas breastfeeding would have a positive impact on child temperament. Mothers' ratings of child's temperament, own psychiatric symptomatology, environmental stresses and family cohesion were obtained in 504 mother-infant dyads via rating scales completed by mothers. Infants were breastfeed (BF), fed soy-based infant formula (SF) or dairy-based infant formula (MF). Linear mixed effect models investigated the relationship of variables on child's temperament while controlling for significant covariates and repeated measurements. Mothers in this study did not endorse clinical-level psychiatric symptomatology; however, when adjusted for significant covariates, higher psychiatric symptomatology significantly correlated with environmental stressors, impaired family dynamics and elevations in temperament ratings of infants' adaptability and mood. There were no lasting differences for temperament between feeding groups. However, some significant transient increases in rhythmicity and adaptability were found between SF and BF children. Positive relationships between family environment stressors and maternal psychiatric ratings were found. Transient differences were found in child temperament based upon feeding method.

CLARA: A cluster-based node correlation for sampling rate adaptation and fault tolerance in sensor networks

Recently, wireless sensor networks (WSNs) have been proven as an efficient and low-cost solution for monitoring various kind of applications. However, the massive amount of data collected and transmitted by the sensor nodes, which are mostly redundant, will quickly consume their limited battery power, which is sometimes difficult to replace or recharge. Although the huge efforts made by researchers to solve such problem, most of the proposed techniques suffer from their accuracy and their complexity, which is not suitable for limited-resources sensors. Therefore, designing new data reduction techniques to reduce the raw data collected in such networks is becoming essential to increase their lifetime. In this paper, we propose a CLuster-based node correlation for sAmpling Rate adaptation and fAult tolerance, abbreviated CLARA, mechanism dedicated to periodic sensor network applications. Mainly, CLARA works on two stages: node correlation and fault tolerance. The first stage introduces a data clustering method that aims to search the correlation among neighboring nodes. Then, it accordingly adapts their sensing frequencies in a way to reduce the amount of data collected in such networks while preserving the information integrity at the sink. In the second stage, a fault tolerance model is proposed that allows the sink to regenerate the raw sensor data based on two methods: moving average (MA) and exponential smoothing (ES). We demonstrated the efficiency of our technique through both simulations and experiments. The best obtained results show that the first stage can reduce the sensor sampling rate, and accordingly the sensor energy, up to 64% while the second stage can accurately regenerate the raw data with an error loss less than 0.15.

Sigmoidal learning rate optimizer for deep neural network training using a two-phase adaptation approach

The optimization of machine learning models is an open research question since an optimal procedure to change the learning rate throughout training is still unknown. Manually defining a learning rate schedule involves troublesome, time-consuming try-and-error procedures to determine hyperparameters, such as learning rate decay epochs and learning rate decay rates, in order to navigate the intricate landscape of loss functions. Although adaptive learning rate optimizers automatize this process, recent studies suggest they may produce overfitting and reduce performance compared to fine-tuned learning rate schedules. Considering that the new machine learning loss function approaches present landscapes with much more saddle points than local minima, we proposed the Training Aware Sigmoidal Optimizer (TASO). This automated two-phase learning rate adaptation mechanism significantly reduces the need for manual hyperparameter tuning. The first phase uses a high learning rate to quickly traverse the numerous saddle points in the error surface, while the second phase uses a low learning rate to gradually approach the center of the local minimum previously found. We compared the proposed approach with commonly used adaptive learning rate schedules such as Adam, RMSProp, and Adagrad. The validation experiments were performed in image and text datasets and showed that TASO outperformed all competing methods in optimal (i.e., performing hyperparameter validation) and suboptimal (i.e., using default hyperparameters) scenarios. In our benchmark tests, TASO demonstrated promising performance, achieving an 8.32% increase in accuracy and a significant 46.62% decrease in training loss on average across various datasets and models. This remarkable performance positions TASO ahead of well-established adaptive optimizers, suggesting higher effectiveness and consistency in performance.

Efficient adaptive learning rate for convolutional neural network based on quadratic interpolation egret swarm optimization algorithm

Convolutional neural network (CNN) has recently become popular for addressing multi-domain image classification. However, most existing methods frequently suffer from poor performance, especially in performance and convergence for various datasets. Herein, we have proposed an algorithm for multi-domain image classification by introducing a novel adaptive learning rate rule to the conventional CNN. Specifically, we adopt the CNN to extract rich feature representations. Given that the hyperparameters of the learning rate have a positive effect on the prediction error, the Egret Swarm Optimization Algorithm (ESOA) is introduced to update the learning rate, which can jump out of local extrema during exploration. Therefore, combined with quadratic interpolation, the objective function can be approximated by a polynomial, thereby improving its prediction accuracy. To verify the robustness of the proposed algorithm, we conducted comprehensive experiments in five domain public datasets to fulfil the task of image classification. Meanwhile, the highest accuracy rate of 97.15 % was obtained on the test set. The performances of our method on 24 benchmark functions (CEC2017 and CEC2022) are compared with Particle Swarm Optimization (PSO), Genetic Algorithm(GA), Whale Optimization Algorithm(WOA), Catch Fish Optimization Algorithm(CFOA), GOOSE Algorithm(GO) and ESOA. In two benchmark sets, the performance metric values of our algorithm rank no. 1, especially in all unimodal functions in contrast with other baseline algorithms.