Efficient Adaptive Algorithm Research Articles

Adaptive search for broad attention based vision transformers

Vision Transformer (ViT) has prevailed among computer vision tasks for its powerful capability of image representation recently. Frustratingly, the manual design of efficient architectures for ViTs can be laborious, often involving repetitive trial and error processes. Furthermore, the exploration of lightweight ViTs remains limited, resulting in inferior performance compared to convolutional neural networks. To tackle these challenges, we propose Adaptive Search for Broad attention based Vision Transformers, called ASB, which automates the design of efficient ViT architectures by utilizing the broad search space and an adaptive evolutionary algorithm. The broad search space facilitates the exploration of a novel connection paradigm, enabling more comprehensive integration of attention information to improve ViT performance. Additionally, an efficient adaptive evolutionary algorithm is developed to efficiently explore architectures by dynamically learning the probability distribution of candidate operators. Our experimental results demonstrate that the adaptive evolution in ASB efficiently learns excellent lightweight models, achieving a 55% improvement in convergence speed over traditional evolutionary algorithms. Moreover, the effectiveness of ASB is validated across several visual tasks. For instance, on ImageNet classification, the searched model attains a performance of 77.8% with 6.5M parameters and outperforms state-of-the-art models, including EfficientNet and EfficientViT networks. On mobile COCO panoptic segmentation, our approach delivers 43.7% PQ. On mobile ADE20K semantic segmentation, our method attains 40.9% mIoU. The code and pre-trained models will be available soon in ASB-Code.

Time-asymmetric fluctuation theorem and efficient free-energy estimation.

The free-energy difference ΔF between two high-dimensional systems is notoriously difficult to compute but very important for many applications such as drug discovery. We demonstrate that an unconventional definition of work introduced by Vaikuntanathan and Jarzynski (2008) satisfies a microscopic fluctuation theorem that relates path ensembles that are driven by protocols unequal under time reversal. It has been shown before that counterdiabatic protocols-those having additional forcing that enforces the system to remain in instantaneous equilibrium, also known as escorted dynamics or engineered swift equilibration-yield zero-variance work measurements for this definition. We show that this time-asymmetric microscopic fluctuation theorem can be exploited for efficient free-energy estimation by developing a simple (i.e., neural-network free) and efficient adaptive time-asymmetric protocol optimization algorithm that yields ΔF estimates that are orders of magnitude lower in mean squared error than the generic linear interpolation protocol with which it is initialized.

Sequential optimal experimental design for vapor-liquid equilibrium modeling

We propose a general methodology of sequential locally optimal design of experiments for explicit or implicit nonlinear models, as they abound in chemical engineering and, in particular, in vapor-liquid equilibrium modeling. As a sequential design method, our method iteratively alternates between performing experiments, updating parameter estimates, and computing new experiments. Specifically, our sequential design method computes a whole batch of new experiments in each iteration and this batch of new experiments is designed in a two-stage locally optimal manner. In essence, this means that in every iteration the combined information content of the newly proposed experiments and of the already performed experiments is maximized. In order to solve these two-stage locally optimal design problems, a recent and efficient adaptive discretization algorithm is used. We demonstrate the benefits of the proposed methodology on the example of the parameter estimation for the non-random two-liquid model for narrow azeotropic vapor-liquid equilibria. As it turns out, our sequential optimal design method requires substantially fewer experiments than traditional factorial design to achieve the same model precision and prediction quality. Consequently, our method can contribute to a substantially reduced experimental effort in vapor-liquid equilibrium modeling and beyond.

Efficient and Robust Adaptive Beamforming Based on Coprime Array Interpolation

Unlike uniform linear arrays (ULAs), coprime arrays require fewer physical sensors yet provide higher degrees of freedom (DOF) and larger array apertures. However, due to the existence of “holes” in the differential co-array, the target detection performance deteriorates, especially in adaptive beamforming. To address these challenges, this paper proposes an efficient and robust adaptive beamforming algorithm leveraging coprime array interpolation. The algorithm eliminates unwanted signals and uses the Gauss–Legendre quadrature method to reconstruct an Interference-plus-Noise Covariance Matrix (INCM), thereby obtaining the beamforming coefficients. Unlike previous techniques, we utilize a virtual interpolated ULA to expand the aperture, enabling the acquisition of a high-dimensional covariance matrix. Additionally, a projection matrix is constructed to eliminate unwanted signals from the received data, greatly enhancing the accuracy of INCM reconstruction. To address the high computational complexity of integral operations used in most INCM reconstruction algorithms, we propose an approximation based on the Gauss–Legendre quadrature, which reduces the computational load while maintaining accuracy. This algorithm avoids the array aperture loss caused by using only the ULA segment in the difference co-array and improves the accuracy of INCM reconstruction. Simulation and experimental results show that the performance of the proposed algorithm is superior to the compared beamformers and is closer to the optimal beamformer in various scenarios.

Modeling and simulation of chemo-elasto-plastically coupled battery active particles

AbstractAs an anode material for lithium-ion batteries, amorphous silicon offers a significantly higher energy density than the graphite anodes currently used. Alloying reactions of lithium and silicon, however, induce large deformation and lead to volume changes up to 300%. We formulate a thermodynamically consistent continuum model for the chemo-elasto-plastic diffusion-deformation behavior of amorphous silicon and it’s alloy with lithium based on finite deformations. In this paper, two plasticity theories, i.e. a rate-independent theory with linear isotropic hardening and a rate-dependent one, are formulated to allow the evolution of plastic deformations and reduce occurring stresses. Using modern numerical techniques, such as higher order finite element methods as well as efficient space and time adaptive solution algorithms, the diffusion-deformation behavior resulting from both theories is compared. In order to further increase the computational efficiency, an automatic differentiation scheme is used, allowing for a significant speed up in assembling time as compared to an algorithmic linearization for the global finite element Newton scheme. Both plastic approaches lead to a more heterogeneous concentration distribution and to a change to tensile tangential Cauchy stresses at the particle surface at the end of one charging cycle. Different parameter studies show how an amplification of the plastic deformation is affected. Interestingly, an elliptical particle shows only plastic deformation at the smaller half axis. With the demonstrated efficiency of the applied methods, results after five charging cycles are also discussed and can provide indications for the performance of lithium-ion batteries in long term use.

Sigmoni: classification of nanopore signal with a compressed pangenome index.

Improvements in nanopore sequencing necessitate efficient classification methods, including pre-filtering and adaptive sampling algorithms that enrich for reads of interest. Signal-based approaches circumvent the computational bottleneck of basecalling. But past methods for signal-based classification do not scale efficiently to large, repetitive references like pangenomes, limiting their utility to partial references or individual genomes. We introduce Sigmoni: a rapid, multiclass classification method based on the r-index that scales to references of hundreds of Gbps. Sigmoni quantizes nanopore signal into a discrete alphabet of picoamp ranges. It performs rapid, approximate matching using matching statistics, classifying reads based on distributions of picoamp matching statistics and co-linearity statistics, all in linear query time without the need for seed-chain-extend. Sigmoni is 10-100× faster than previous methods for adaptive sampling in host depletion experiments with improved accuracy, and can query reads against large microbial or human pangenomes. Sigmoni is the first signal-based tool to scale to a complete human genome and pangenome while remaining fast enough for adaptive sampling applications. Sigmoni is implemented in Python, and is available open-source at https://github.com/vshiv18/sigmoni.

Dispatchable region for distributed renewable energy generation in reconfigurable AC–DC distribution networks with soft open points

The dispatchable region (DR) describes the ability of a power system to adapt to the variability of distributed renewable energy (DRE) generation. Switchable devices such as soft open points (SOPs) and tie switches enable the AC–DC distribution network to flexibly cope with DRE power fluctuations. Currently, research on DRs seldom considers the effects of SOPs and reconfiguration with tie switches in AC–DC distribution networks, which prevents accurate evaluation of DR for this type of network. To bridge this gap, this paper formulates DRs for DRE generation in an AC–DC distribution network considering SOPs and reconfiguration with tie switches. The original model for deriving DRs is typically nonconvex, and the DR boundaries are intractable because of nonlinear power flow equations and binary variables of tie switches. Accordingly, this paper employs the outer polyhedral approximation and convex hull relaxation to approximate the original DR. An efficient adaptive constraint generation algorithm is developed to search for boundaries of the approximated DR. The numerical results show that the proposed model can effectively adapt to the DR of reconfigurable AC–DC distribution networks with SOPs and verify the accuracy and computational efficiency of the proposed method.

DJADE: a reliable and efficient adaptive differential evolution algorithm with selection pressure control

Our main contribution is the proposition of a new and explicit way of adapting the parental selection pressure for Differential Evolution algorithm (DE) and its experimentation on an adaptive version of differential evolution algorithm named JADE in order to improve its reliability without diminishing its efficiency. The new algorithm named DJADE employs a genotypic population diversity measure (a measure of population convergence) to control the selection pressure parameter. A mechanism that increases the parental selection pressure as the search progresses and in the same time serves as a mean to detect when the algorithm begins to converge (the "convergence phase") is presented. In addition, DJADE is a novel adaptive DE, which adapts crossover, mutation and selection pressure parameters. Moreover, the coefficient K of greediness of the mutation scheme DE/current-to- best/ is also adapted. The motivation for the parameter adaptation is to obtain an algorithm that is capable to handle various problems with different characteristics, notably difficult multimodal and/or nonseparable functions. No case of premature convergence is observed during the experiments conducted on 13 classical functions test at 05 and 30 dimensions. The comparison with classical JADE and Classical DE are presented. DJADE is not only very efficient but also reliable on all functions tested.

Anti-Windup Adaptive Look-Up Table Algorithms with Application to Data-Driven Engine Controls

Abstract This technical brief presents anti-windup adaptation algorithms for a look-up table, widely used in data-driven engine control systems to accurately model complex features while minimizing computational demand. Engine control systems are prone to uncertain variations due to aging, faults, and manufacturing tolerances, which can impact performance and emissions unless effectively managed. Therefore, there is a growing demand for adaptive features in these systems to maintain robust performance and emissions over their lifespan. This study develops computationally efficient adaptive look-up table algorithms using anti-windup recursive parameter estimation and covariance matrix resetting, ensuring robust and rapid adaptation under various operating conditions. The effectiveness of these algorithms is demonstrated through adapting an engine-out nitrogen oxides concentration map, which is crucial for tailpipe emission controls in compression-ignition engines.

Combinatorial optimization methods for yarn dyeing planning

AbstractManaging yarn dyeing processes is one of the most challenging problems in the textile industry due to its computational complexity. This process combines characteristics of multidimensional knapsack, bin packing, and unrelated parallel machine scheduling problems. Multiple customer orders need to be combined as batches and assigned to different shifts of a limited number of machines. However, several practical factors such as physical attributes of customer orders, dyeing machine eligibility conditions like flotte, color type, chemical recipe, and volume capacity of dye make this problem significantly unique. Furthermore, alongside its economic aspects, minimizing the waste of natural resources during the machine changeover and energy are sustainability concerns of the problem. The contradictory nature of these two makes the planning problem multi-objective, which adds another complexity for planners. Hence, in this paper, we first propose a novel mathematical model for this scientifically highly challenging yet very practical problem from the textile industry. Then we propose Adaptive Large Neighbourhood Search (ALNS) algorithms to solve industrial-size instances of the problem. Our computational results show that the proposed algorithm provides near-optimal solutions in very short computational times. This paper provides significant contributions to flexible manufacturing research, including a mixed-integer programming model for a novel industrial problem, providing an effective and efficient adaptive large neighborhood search algorithm for delivering high-quality solutions quickly, and addressing the inefficiencies of manual scheduling in textile companies; reducing a time-consuming planning task from hours to minutes.

An efficient multi-objective adaptive large neighborhood search algorithm for solving a disassembly line balancing model considering idle rate, smoothness, labor cost, and energy consumption

The concept of green manufacturing emphasizes the importance of product disassembly in achieving energy-efficient recycling and remanufacturing operations. Disassembly line balancing (DLB) is a critical component of the product disassembly process, wherein a set of tasks must be allocated to workstations for disassembly. This study proposes a multi-objective DLB model that aims to minimize multiple conflicting objectives simultaneously including idle rate, smoothness, labor cost, and energy consumption. A key innovation of this study involves the creation of a tailored adaptive large neighborhood search (ALNS) algorithm which is one of the first studies in the literature on product disassembly algorithms. The developed ALNS employs efficient construction and destruction heuristics to solve the proposed multi-objective DLB problem. The ALNS algorithm aims to destroy and repair solutions effectively, while a local search procedure helps it escape from local optimum solutions. The provided DLB model is effectively solved by applying the proposed ALNS algorithm to the disassembly process of a turbine reducer. The obtained results from this application serve as a compelling demonstration of the efficiency and effectiveness of the proposed approach. Furthermore, comparisons conducted with various state-of-the-art algorithms using small and large instance sets consistently highlight the superiority of the proposed ALNS approach.

Computationally efficient robust adaptive filtering algorithm based on improved minimum error entropy criterion with fiducial points

Recently, there has been a strong interest in the minimum error entropy (MEE) criterion derived from information theoretic learning, which is effective in dealing with the multimodal non-Gaussian noise case. However, the kernel function is shift invariant resulting in the MEE criterion being insensitive to the error location. An existing solution is to combine the maximum correntropy (MC) with MEE criteria, leading to the MEE criterion with fiducial points (MEEF). Nevertheless, the algorithms based on the MEEF criterion usually require higher computational complexity. To remedy this problem, an improved MEEF (IMEEF) criterion is devised, aiming to avoid repetitive calculations of the aposteriori error, and an adaptive filtering algorithm based on gradient descent (GD) method is proposed, namely, GD-based IMEEF (IMEEF-GD) algorithm. In addition, we provide the convergence condition in terms of mean sense, along with an analysis of the steady-state and transient behaviors of IMEEF-GD in the mean-square sense. Its computational complexity is also analyzed. Simulation results demonstrate that the computational requirement of our algorithm does not vary significantly with the error sample number and the derived theoretical model is highly consistent with the learning curve. Ultimately, we employ the IMEEF-GD algorithm in tasks such as system identification, wind signal magnitude prediction, temperature prediction, and acoustic echo cancellation (AEC) to validate the effectiveness of the IMEEF-GD algorithm.

An Improved Fault Diagnosis Algorithm for Highly Scalable Data Center Networks

Fault detection and localization are vital for ensuring the stability of data center networks (DCNs). Specifically, adaptive fault diagnosis is deemed a fundamental technology in achieving the fault tolerance of systems. The highly scalable data center network (HSDC) is a promising structure of server-centric DCNs, as it exhibits the capacity for incremental scalability, coupled with the assurance of low cost and energy consumption, low diameter, and high bisection width. In this paper, we first determine that both the connectivity and diagnosability of the m-dimensional complete HSDC, denoted by HSDCm(m), are m. Further, we propose an efficient adaptive fault diagnosis algorithm to diagnose an HSDCm(m) within three test rounds, and at most N+4m(m−2) tests with m≥3 (resp. at most nine tests with m=2), where N=m·2m is the total number of nodes in HSDCm(m). Our experimental outcomes demonstrate that this diagnosis scheme of HSDC can achieve complete diagnosis and significantly reduce the number of required tests.

A novel fast and efficient adaptive shuffled complex evolution algorithm for model parameter calibration

The research and optimization of hydrological forecasting models are among the most crucial components in the scope of water management and flood protection. Optimizing the calibration of hydrological forecasting models is crucial for forecasting performance. A rapid adaptive Shuffled Complex Evolution (SCE) method called Fast Adaptive SCE (FASCE) is proposed for calibrating model parameters. It builds upon the previously established SCE-UA, known for its effectiveness and robustness in the same calibration context. The robustness of the original SCE-UA is expanded upon, introducing a revised adaptive simplex search to bolster efficiency. Additionally, a new strategy for setting up the initial population base enhances explorative capacities. FASCE’s performance has been assessed alongside numerous methods from prior studies, demonstrating its effectiveness. Initial tests were conducted on a set of functions to assess FASCE’s efficacy. Findings revealed that FASCE could curtail the failure rate by a minimum of 80%, whereas the requirement for function evaluations fell between 30% and 60%. Two hydrological models - Support Vector Machine (SVM) and Xinanjiang rainfall-runoff model were employed to estimate the new algorithm’s performance. No failures were reported, and there was a reduction of at least 30% in function evaluations using FASCE. The outcomes from these studies affirm that FASCE can considerably reduce both the number of failures and the count of function evaluations required to reach the global maximum. Hence, FASCE emerges as a viable substitute for model parameter calibration.

An efficient adaptive Masi entropy multilevel thresholding algorithm based on dynamic programming

Masi entropy multilevel thresholding can utilize additive and non-extensive information in images to effectively segment a complex image. However, the entropy index of Masi entropy cannot be selected automatically, and the time complexity of the multilevel algorithm by exhaustive searching grows exponentially with the increase of the threshold numbers. To address these two problems, an adaptive entropy index selection strategy based on image histogram information is proposed first. To improve the computation efficiency, an efficient solution for the adaptive Masi entropy multilevel thresholding algorithm based on dynamic programming (DP + AMasi) is also proposed. The DP + AMasi algorithm is compared with the Masi entropy multilevel thresholding algorithm by exhaustive search and state-of-the-art metaheuristic algorithms on three benchmark datasets. The effectiveness of the DP + AMasi is verified by fitness function values, Uniformity Measure, Davies Bouldin index, and CPU run time. In addition, the Wilcoxon test is used to analyze the differences between algorithms.

An efficient new adaptive variational mode decomposition algorithm for extracting adventitious lung sounds

The rapid and accurate extraction of adventitious lung sounds (ALSs) from original lung sounds is greatly significant for diagnosing lung diseases and assessing lung condition. Due to interference from heart sounds and various noises, the empirical mode decomposition (EMD), variational mode decomposition (VMD) and some other classical algorithms have poor performance. Here, an efficient new adaptive variational mode decomposition (AVMD) algorithm is proposed to improve the efficiency for extracting ALSs. For minimizing the bandwidth of each extracted mode and reducing frequency aliasing between modes, the proposed algorithm constructs a frequency-constrained model based on VMD. In order to further improve its performance, the algorithm adopts a strategy of gradually extracting each mode to determine the optimal number of modes for avoiding the under-decomposition or over-decomposition, and uses Pool Adjacent Violators (PAV) to automatically initialize the center frequency of each mode to reduce algorithm iteration steps. Based on the above improved characteristics, the experiments presented in this paper, which conducted on the artificial simulation signal and 102 groups of lung sounds from 20 patients with severe pulmonary diseases, highlight the proposed AVMD's superiority over the other five algorithms when comprehensively considering both decomposition quality and computational efficiency. Furthermore, the above results of 102 groups of lung sounds are statistically analyzed to obtain the characteristic frequency distribution of ALSs, which can provide important references for clinicians and demonstrate the potential application of the proposed algorithm in the diagnosis of pulmonary diseases.

An efficient adaptive routing algorithm for the Co-optimization of fault tolerance and congestion awareness based on 3D NoC

In this paper, we propose an adaptive, fault-tolerant, and congestion-aware (AFTC) routing algorithm. It improves the overall network performance of three-dimensional Network-on-Chip (3D NoC) in terms of deadlock-free, adaptability, fault tolerance, and balanced traffic. Firstly, the algorithm proposed 12 turn models to avoid deadlock problems. Then, the adaptability of the algorithm is improved by using the three-dimensional diagonal region division method. At the same time, the AFTC can reduce the number of turn-around steps during fault tolerance. Finally, the AFTC can balance traffic by setting different congestion thresholds. Compared with Vertical-Mesh-Conscious-Dynamic (VMCD) algorithm in terms of average packet delay and throughput, the AFTC algorithm can improve the throughput by an average of 27.29 % and 18.62 % while reducing the average delay time in different fault rates under uniform traffic and hotspot traffic patterns, respectively.

An Iterative Wiener Filter Based on a Fourth-Order Tensor Decomposition

This work focuses on linear system identification problems in the framework of the Wiener filter. Specifically, it addresses the challenging identification of systems characterized by impulse responses of long length, which poses significant difficulties due to the existence of large parameter space. The proposed solution targets a dimensionality reduction of the problem by involving the decomposition of a fourth-order tensor, using low-rank approximations in conjunction with the nearest Kronecker product. In addition, the rank of the tensor is controlled and limited to a known value without involving any approximation technique. The final estimate is obtained based on a combination of four (shorter) optimal filters, which are alternatively iterated. As a result, the designed iterative Wiener filter outperforms the traditional counterpart, being more robust to the accuracy of the statistics’ estimates and/or noisy conditions. In addition, simulations performed in the context of acoustic echo cancellation indicate that the proposed iterative Wiener filter that exploits this fourth-order tensor decomposition achieves better performance as compared to some previously developed solutions based on lower decomposition levels. This study could further lead to the development of computationally efficient tensor-based adaptive filtering algorithms.

An efficient adaptive large neighborhood search algorithm based on heuristics and reformulations for the generalized quadratic assignment problem

Operations Research (OR) analytics play a crucial role in optimizing decision-making processes for real-world problems. The assignment problem, with applications in supply chains, healthcare logistics, and production scheduling, represents a prominent optimization challenge. This paper focuses on addressing the Generalized Quadratic Assignment Problem (GQAP), a well-known NP-hard combinatorial optimization problem. To tackle the GQAP, we propose an OR analytical approach that incorporates efficient relaxations, reformulations, heuristics, and a metaheuristic algorithm. Initially, we employ the Reformulation Linearization Technique (RLT) to generate various linear relaxation models, carefully selecting the most efficient ones. Building upon this foundation, we introduce a robust reformulation based on Benders Decomposition (BD), which serves as the basis for an iterative optimization algorithm applied to the GQAP. Furthermore, we develop a constructive heuristic algorithm to identify near-optimal solutions, followed by an enhancement utilizing an Adaptive Large Neighborhood Search (ALNS) metaheuristic algorithm. This ALNS algorithm is enhanced through the integration of a tabu list derived from Tabu Search (TS) and a decision rule inspired by Simulated Annealing (SA). To validate our approach and evaluate its performance, we conduct a comparative analysis against state-of-the-art algorithms documented in the literature. This comparison confirms the significant improvements achieved in terms of solution quality and computational efficiency through our proposed methodology. These advancements contribute to the state of the art in solving the GQAP and hold the potential to enhance decision-making processes in a wide range of domains. Our methodology demonstrates remarkable improvements in solution quality and computational efficiency when compared to existing approaches, as evidenced by the comparative results with state-of-the-art algorithms. The potential implications of our research extend to optimizing decision-making processes in diverse fields, rendering it highly relevant and impactful.

Faster Adaptive Federated Learning

Federated learning has attracted increasing attention with the emergence of distributed data. While extensive federated learning algorithms have been proposed for the non-convex distributed problem, the federated learning in practice still faces numerous challenges, such as the large training iterations to converge since the sizes of models and datasets keep increasing, and the lack of adaptivity by SGD-based model updates. Meanwhile, the study of adaptive methods in federated learning is scarce and existing works either lack a complete theoretical convergence guarantee or have slow sample complexity. In this paper, we propose an efficient adaptive algorithm (i.e., FAFED) based on the momentum-based variance reduced technique in cross-silo FL. We first explore how to design the adaptive algorithm in the FL setting. By providing a counter-example, we prove that a simple combination of FL and adaptive methods could lead to divergence. More importantly, we provide a convergence analysis for our method and prove that our algorithm is the first adaptive FL algorithm to reach the best-known samples O(epsilon(-3)) and O(epsilon(-2)) communication rounds to find an epsilon-stationary point without large batches. The experimental results on the language modeling task and image classification task with heterogeneous data demonstrate the efficiency of our algorithms.