Efficient Optimization Algorithm Research Articles

The dual-competitive parallel kinetic model with an efficient optimization algorithm for describing polymer pyrolysis accurately

The dual-competitive parallel kinetic model with an efficient optimization algorithm for describing polymer pyrolysis accurately

Graph structure learning layer and its graph convolution clustering application

To learn the embedding representation of graph structure data corrupted by noise and outliers, existing graph structure learning networks usually follow the two-step paradigm, i.e., constructing a “good” graph structure and achieving the message passing for signals supported on the learned graph. However, the data corrupted by noise may make the learned graph structure unreliable. In this paper, we propose an adaptive graph convolutional clustering network that alternatively adjusts the graph structure and node representation layer-by-layer with back-propagation. Specifically, we design a Graph Structure Learning layer before each Graph Convolutional layer to learn the sparse graph structure from the node representations, where the graph structure is implicitly determined by the solution to the optimal self-expression problem. This is one of the first works that uses an optimization process as a Graph Network layer, which is obviously different from the function operation in traditional deep learning layers. An efficient iterative optimization algorithm is given to solve the optimal self-expression problem in the Graph Structure Learning layer. Experimental results show that the proposed method can effectively defend the negative effects of inaccurate graph structures. The code is available at https://github.com/HeXiax/SSGNN.

Let the Data Choose: Flexible and Diverse Anchor Graph Fusion for Scalable Multi-View Clustering

In the past few years, numerous multi-view graph clustering algorithms have been proposed to enhance the clustering performance by exploring information from multiple views. Despite the superior performance, the high time and space expenditures limit their scalability. Accordingly, anchor graph learning has been introduced to alleviate the computational complexity. However, existing approaches can be further improved by the following considerations: (i) Existing anchor-based methods share the same number of anchors across views. This strategy violates the diversity and flexibility of multi-view data distribution. (ii) Searching for the optimal anchor number within hyper-parameters takes much extra tuning time, which makes existing methods impractical. (iii) How to flexibly fuse multi-view anchor graphs of diverse sizes has not been well explored in existing literature. To address the above issues, we propose a novel anchor-based method termed Flexible and Diverse Anchor Graph Fusion for Scalable Multi-view Clustering (FDAGF) in this paper. Instead of manually tuning optimal anchor with massive hyper-parameters, we propose to optimize the contribution weights of a group of pre-defined anchor numbers to avoid extra time expenditure among views. Most importantly, we propose a novel hybrid fusion strategy for multi-size anchor graphs with theoretical proof, which allows flexible and diverse anchor graph fusion. Then, an efficient linear optimization algorithm is proposed to solve the resultant problem. Comprehensive experimental results demonstrate the effectiveness and efficiency of our proposed framework. The source code is available at https://github.com/Jeaninezpp/FDAGF.

Material parameter optimization of flax/epoxy composite laminates under low-velocity impact

Since natural fibers have great potential as an alternative to synthetic fibers when the components are impacted at low energies, their mechanical properties under different types of loads need to be investigated. This can be accomplished by using finite element analysis, which is based on the definition of numerical models that reproduce the objects of the physical phenomenon under study. In defining these models, many parameters of the material cards are determined by experimental tests. However, experiments are time-consuming and costly, and it is not always possible to perform all the necessary tests to determine the values for all unknown parameters. For this purpose, the trial-and-error method is usually used. In this work, we present an optimization procedure for predicting the behavior of flax/epoxy composite laminates under low-velocity impact, using the LS-DYNA solver for numerical simulation. The study aims at identifying the values of relevant parameters that allow for predicting the experimental force–displacement trend as accurately as possible and reproducing the damage mechanisms numerically. Each step of the optimization flow is performed with the external tool LS-OPT, using dynamic time warping as a similarity measure to efficiently handle noise. For this purpose, we use the Efficient Global Optimization algorithm, a strategy based on surrogate modeling techniques. We address a multi-target scenario, i.e., we consider several energy levels simultaneously, aiming to find an optimal parameter configuration that is less sensitive to variations in impact energy. The results obtained not only demonstrate the potential of surrogate-based optimization to identify material parameters, but also provide a characterization of the studied composite configuration in view of future applications.

Minimax AUC Fairness: Efficient Algorithm with Provable Convergence

The use of machine learning models in consequential decision making often exacerbates societal inequity, in particular yielding disparate impact on members of marginalized groups defined by race and gender. The area under the ROC curve (AUC) is widely used to evaluate the performance of a scoring function in machine learning, but is studied in algorithmic fairness less than other performance metrics. Due to the pairwise nature of the AUC, defining an AUC-based group fairness metric is pairwise-dependent and may involve both intra-group and inter-group AUCs. Importantly, considering only one category of AUCs is not sufficient to mitigate unfairness in AUC optimization. In this paper, we propose a minimax learning and bias mitigation framework that incorporates both intra-group and inter-group AUCs while maintaining utility. Based on this Rawlsian framework, we design an efficient stochastic optimization algorithm and prove its convergence to the minimum group-level AUC. We conduct numerical experiments on both synthetic and real-world datasets to validate the effectiveness of the minimax framework and the proposed optimization algorithm.

Graph attentive matrix factorization for social recommendation

AbstractRecommender systems commonly encounter with the problems of data sparsity and cold start. Recently, social recommendation has emerged with the rapid expansion of social platforms, offering an opportunity to alleviate such two obstacles. Nevertheless, there are still two key limitations in existing studies. From the perspective of model design, previous social recommenders only consider the influence of a user's direct friends or uniformly treat the influences from different friends. From the perspective of model learning, most of them apply a sampling‐based optimization strategy, which requires high‐quality positive and negative samples. In light of the aforementioned limitations, we propose a new probabilistic method, named Graph Attentive Matrix Factorization (GAMF). Our method not only explicitly captures high‐order social relationships, but also adopts an attention mechanism to automatically pick up different influences between friends. Moreover, we develop an efficient optimization algorithm to learn model parameters in a non‐sampling manner. Extensive experiments on four large‐scale datasets show the superiority of GAMF over state‐of‐the‐art recommenders, especially under the cold start scenario.

Dependent tasks offloading in mobile edge computing: A multi-objective evolutionary optimization strategy

Due to the proliferation of applications such as virtual reality and online games with high real-time requirements, Mobile Edge Computing (MEC) has become a promising computing paradigm that can improve user experience and reduce the task offloading latency. The cloud–edge-end collaborative offloading further addresses the problem of insufficient computing resources of edge servers owing to large-scale computing-intensive applications in MEC. However, existing offloading solutions often ignore the important factor of economic cost, making it hard for these solutions to achieve a sustainable cloud–edge-end collaborative computation. To this end, this paper considers a multi-user multi-server, cloud–edge-end collaborative offloading scenario in the presence of dependent offloading tasks for the sake of maximizing rewards and minimizing latency. Each user issues a computing-intensive application consisting of multiple dependent tasks, which are offloaded collaboratively by various computational resources. With the goal of maximizing the yield of offloading for users and server providers, a multi-objective optimization problem of joint task offloading and execution rewards is studied. Technically, a multivariate multi-objective optimization problem with three objectives is modeled. An efficient multi-objective evolutionary optimization algorithm based on MOEA/D is then developed to solve the latency minimization and reward maximization problems. Extensive simulation results verify the effectiveness of the algorithm and illustrate that the proposed algorithm can significantly improve user offloading benefits. In addition, a scalability evaluations of our proposed algorithm is conducted for demonstrating its feasibility in large-scale task offloading scenarios.

Optimization and Learning With Randomly Compressed Gradient Updates.

Gradient descent methods are simple and efficient optimization algorithms with widespread applications. To handle high-dimensional problems, we study compressed stochastic gradient descent (SGD) with low-dimensional gradient updates. We provide a detailed analysis in terms of both optimization rates and generalization rates. To this end, we develop uniform stability bounds for CompSGD for both smooth and nonsmooth problems, based on which we develop almost optimal population risk bounds. Then we extend our analysis to two variants of SGD: batch and mini-batch gradient descent. Furthermore, we show that these variants achieve almost optimal rates compared to their high-dimensional gradient setting. Thus, our results provide a way to reduce the dimension of gradient updates without affecting the convergence rate in the generalization analysis. Moreover, we show that the same result also holds in the differentially private setting, which allows us to reduce the dimension of added noise with "almost free" cost.

Interpretable cost-sensitive regression through one-step boosting

In most practical prediction problems, such as regression and classification, the different types of prediction errors are not equally costly in the decision-making process. Although there exist numerous real-world cost-sensitive regression problems, ranging from loan charge-off forecasting to house price predictions, the literature on cost-sensitive learning mainly focuses on classification and only a few solutions are proposed for regression problems. These regressions are typically characterized by an asymmetric cost structure, where over- and underpredictions of a similar magnitude face vastly different costs. In this paper, we present a one-step boosting method (OSB) for cost-sensitive regression. The proposed methodology leverages a secondary learner to incorporate cost-sensitivity into an already trained cost-insensitive regression model. The secondary learner is defined as a linear function of certain variables deemed interesting for cost-sensitivity. These variables do not necessarily need to be the same as in the already trained model. An efficient optimization algorithm is achieved through iteratively reweighted least squares using the asymmetric cost function. The obtained results become interpretable through bootstrapping, enabling decision makers to distinguish important variables for cost-sensitivity as well as facilitating statistical inference. Applying different cost functions and various initial cost-insensitive learning methods on several public datasets consistently yields a significant reduction in the average misprediction cost, illustrating the excellent performance of our approach.

High-Throughput Synthesis and Machine Learning Assisted Design of Photodegradable Hydrogels.

Due to the large chemical space, the design of functional and responsive soft materials poses many challenges but also offers a wide range of opportunities in terms of the scope of possible properties. Herein, an experimental workflow for miniaturized combinatorial high-throughput screening of functional hydrogel libraries is reported. The data created from the analysis of the photodegradation process of more than 900 different types of hydrogel pads are used to train a machine learning model for automated decision making. Through iterative model optimization based on Bayesian optimization, a substantial improvement in response properties is achieved and thus expanded the scope of material properties obtainable within the chemical space of hydrogels in the study. It is therefore demonstrated that the potential of combining miniaturized high-throughput experiments with smart optimization algorithms for cost and time efficient optimization of materials properties.

Efficient prestack time migration velocity analysis: A correlation‐based global optimization approach

AbstractAn accurate migration velocity model is vital to seismic imaging. Conventional time‐domain migration velocity analysis techniques commonly obtain the velocity model by iteratively picking the velocity spectra of common reflection point gathers or scanning selection, which is usually laborious and computationally expensive. To solve these issues, we propose a more effective migration velocity analysis method for prestack time migration. This method is based on cross‐correlation stacked time shift functions of local migrated gathers and uses the very fast simulated annealing algorithm to semi‐automatically invert the migration velocity parameters. This new correlation‐based objective function can catch subtle bending features of the reflector in the local migrated gather. The proposed approach applies an efficient global optimization algorithm that does not depend on the initial parameter value and is easy to extend to the multiparameter complex case. Moreover, the strategy of using local migrated gathers can incorporate prior information in the inversion to avoid the effects of misidentifying reflection events and the interference of multiples. Furthermore, applying a dynamic parameter constraint strategy for the very fast simulated annealing algorithm improves the convergence speed. Both synthetic and real data examples demonstrate that the proposed migration velocity analysis method can efficiently build an accurate time migration velocity model, which can also provide a good initial velocity model for depth migration.

Fast Homotopy for Spacecraft Rendezvous Trajectory Optimization with Discrete Logic

This paper presents a computationally efficient optimization algorithm for solving nonconvex optimal control problems that involve discrete logic constraints. Traditional solution methods require binary variables and mixed-integer programming (MIP), which is prohibitively slow and computationally expensive. This paper proposes a faster and computationally cheaper algorithm that can produce locally optimal solutions in seconds. This is achieved by blending sequential convex programming and numerical continuation into a single iterative solution process. The algorithm approximates discrete logic constraints with smooth functions and uses a homotopy parameter to control the accuracy of this approximation. The homotopy parameter is updated such that, by the time the algorithm converges, the smooth approximations enforce the exact discrete logic. The effectiveness of this approach is numerically demonstrated for a realistic rendezvous scenario inspired by the Apollo Transposition and Docking maneuver. In less than 15 s of cumulative solver time, the algorithm finds a fuel-minimizing trajectory that obeys the following discrete logic constraints: thruster minimum impulse-bit, range-triggered approach cone, and range- triggered plume impingement. The optimized trajectory uses significantly less fuel than reported NASA design targets.

Discrete limited attentional collaborative filtering for fast social recommendation

Over the last few years, social recommendation has attracted tremendous attention due to the ever-growing online social platform such as Twitter and Facebook. However, as the number of users increases rapidly, recommendation efficiency has become the bottleneck of many existing social recommender systems due to the computation and storage of real-valued models. For addressing the efficiency problem, recent researches resolve it by introducing hashing technique into social recommender systems. By mapping real values to discrete values, the computational speed is guaranteed as well as the storage cost is reduced. Nevertheless, these methods suffer from two critical limitations: (1) The inevitable quantization loss brought by hash function decreases recommendation accuracy to a certain extent. (2) The original social relations contain massive noise that may result in sub-optimal accuracy of recommendation without considering the fact that people can only pay attention to a small number of their friends. Therefore, to tackle the above limitations and have a better tradeoff between accuracy and efficiency, in this paper, we propose a novel social recommendation method called Discrete Limited Attentional Collaborative Filtering (DLACF), which models recommendation objective with limited attention as a constrained mix-integer optimization problem. Since the original problem is NP-hard, we further devise a computationally efficient optimization algorithm to learn the binary codes as well as to estimate the best influential friends. Experimental results conducted on two real-world datasets demonstrate the effectiveness of our proposed model, achieving the averaged improvement of 118.7% and 54.7% compared to state-of-the-art discrete methods.

Application of quantum computing in discrete portfolio optimization

This study proposes a novel and more efficient quantum algorithm for portfolio optimization using quantum combinatorial optimization (QCO) techniques. A recent construction developed in 2021 has sparked the field of financial portfolio optimization through the Quantum Walk Optimization Algorithm (QWOA). In this study, we investigated the complexity and efficiency of quantum optimization algorithms with a special interest in QWOA. The objective is to minimize investment risk by having a good combination of assets in the portfolio. We also focused on reducing the number of iterations while attaining a high-quality resolution through contraction of the solution space to ease computations. The concept of QWOA was extended by constructing a newly outperforming scheme known as the “Quantum Mix Optimization Algorithm (QMOA).” QMOA algorithm codes were provided for the implementation and simulation of numerical results. In addition, the efficiency of QMOA, which is better than the existing QCO algorithms, was discussed. For instance, the least QWOA number of computations required to execute the initial state equation was p > 2, whereas this value was p ≥ 2 in the proposed QMOA.

Fast, Lightweight, and Efficient Cybersecurity Optimization for Tactical–Operational Management

The increase in frequency and complexity of cyberattacks has heightened concerns regarding cybersecurity and created an urgent need for organizations to take action. To effectively address this challenge, a comprehensive and integrated approach is required involving a cross-functional cybersecurity workforce that spans tactical and operational levels. In this context there can be various combinations of cybersecurity actions that affect different functional domains and that allow for meeting the established requirements. In these cases, agreement will be needed, but finding high-quality combinations requires analysis from all perspectives on a case-by-case basis. With a large number of cybersecurity factors to consider, the size of the search space of potential combinations becomes unmanageable without automation. To solve this issue, we propose Fast, Lightweight, and Efficient Cybersecurity Optimization (FLECO), an adaptive, constrained, and multi-objective genetic algorithm that reduces the time required to identify sets of high-quality cybersecurity actions. FLECO enables productive discussions on viable solutions by the cross-functional cybersecurity workforce within an organization, fostering managing meetings where decisions are taken and boosting the overall cybersecurity management process. Our proposal is novel in its application of evolutionary computing to solve a managerial issue in cybersecurity and enhance the tactical–operational cybersecurity management process.

A decomposition-based multi-objective Jaya algorithm for lot-streaming job shop scheduling with variable sublots and intermingling setting

This study examined the lot-streaming job shop scheduling problem (LSJSP) with variable sublots and intermingling setting, which has rarely been considered in the literature and is beneficial to real industry settings. Upon variable sublots and intermingling setting, a multi-objective mixed-integer linear programming model (MILP) is formulated to achieve a tradeoff between the shortest tardiness and the minimum number of transferred sublots. Considering the NP-hard characteristic of the problem, the MILP model would get restricted in solving medium and large scale instances. Thus building an efficient multi-objective optimization algorithm is desirable. Motivated by the Jaya algorithm and decomposition idea, we propose a new algorithm framework called a decomposition based multi-objective Jaya algorithm (MOJA/D) to achieve a balance between exploration and exploitation. This MOJA/D framework can provide some guidance in algorithm construction for other multi-objective optimization problems. Following the MOJA/D framework, a specific MOJA/D metaheuristic is further designed for coping with the proposed problem. First, a novel forward and backward decoding strategy is designed to implement coevolution among different sub-problems and enhance the population diversity. Second, the Jaya updating mechanism is designed based on problem knowledge to guide the solutions moving towards the best solution and away from the worst solution. Moreover, considering the tradeoff between two conflicting objectives, the problem-specific local search strategies are designed and integrated with the neighborhood based local search strategies to enhance exploitation in the search process. The influence of parameter setting on MOJA/D is investigated by using design of experiments. The statistical results from extensive computational experiments demonstrate the effectiveness of the specific algorithm designs and show that the MOJA/D metaheuristic is superior to the existing algorithms in solving the proposed problem.

Efficient Random Forest Algorithm for Multi-objective Optimization in Software Defect Prediction

The Software Defect Prediction (SDP) process provides reliable software by identifying defect-prone modules before the testing stage. It efficiently and effectively utilizes quality assurance resources. Most predictive models are trained on historical data which belong to the same project or comparable project. These models show satisfactory performance as they utilize similar settings to the considered projects. But the limitation of these models is that they are effective only if there are adequate historical data to train a predictive model. In reality, however, such historical data are minimal for some projects and absent for new projects. The defect prediction in such projects which lack historical data can be accomplished by training prediction models on different project data. This process is known as Cross-Project Defect Prediction (CPDP). Software defect datasets also suffered from class imbalance issues which further degrades the model’s performance. In this research work, the authors have proposed a Multi-Objective Random Forest (MO-RF) algorithm with a data resampling technique to minimize the probability of false alarms, to maximize the probability of detection and to overcome the class imbalance problem. The study also evaluates the performance of other prediction models. The proposed method has shown percentage improvement (in terms of AUC) of 2.78 and 3.46 over MONB and MONBNN, respectively.

Simulation-based optimization considering energy consumption for assisted station locations to enhance flex-route transit

As an emerging mode of public transit, flex-route transit (FRT) is regarded as the key to reducing energy consumption and promoting sustainable development in low-density suburban cities. However, its performance in cases with unexpectedly high demand levels is usually unsatisfactory. To address this issue, a simulation-based optimization method is proposed for assisted station locations in FRT and to reduce the system costs, including that for energy consumption. First, considering the uncertain travel demands of passengers, this location problem is modelled as a simulation-based optimization (SO) problem based on Monte Carlo simulation (MCS). Then, a multiple metamodel-based efficient global optimization (MMEGO) algorithm is proposed to effectively solve the SO problem. The numerical results demonstrate that the proposed MMEGO algorithm can obtain good results with lower computing cost and stronger robustness than counterpart algorithms (EGO and MISK). Finally, simulation tests of FRT based on data from the operation of 320 buses in Shenzhen city and the related OD data are conducted to test the feasibility of the proposed method. The results show that the proposed optimization method can be used to identify suitable locations for assisted stations and reduce the total system cost by 26.6% compared with that for an FRT system with unoptimized assisted station locations.

Asymptotic pinning synchronization of nonlinear multi-agent systems: Its application to tunnel diode circuit

This work proposes a pinning state-feedback control technique for synchronizing non-linear multi-agent systems (MASs) with time delays. A collection of switching-directed graphs describes the communication exchanges between all of the agents. The challenge of asymptotic stability analysis for some error systems is translated into the construction of a leader-following synchronization of the relevant MASs. The closed-loop system could be acquired by building a convenient Lyapunov–Krasovskii functional (LKF) that has two integral terms, and by using Kronecker product qualities combined with matrix inequality techniques. When these conditions are met, a state-feedback pinning controller can be built with linear matrix inequalities (LMIs), which can be derived easily from a number of efficient optimization algorithms. Further, the performance of the proposed control design system is verified based on a tunnel diode circuit (TDC) by numerical simulations.

Surrogate-Assisted Hybrid-Model Estimation of Distribution Algorithm for Mixed-Variable Hyperparameters Optimization in Convolutional Neural Networks.

The performance of a convolutional neural network (CNN) heavily depends on its hyperparameters. However, finding a suitable hyperparameters configuration is difficult, challenging, and computationally expensive due to three issues, which are 1) the mixed-variable problem of different types of hyperparameters; 2) the large-scale search space of finding optimal hyperparameters; and 3) the expensive computational cost for evaluating candidate hyperparameters configuration. Therefore, this article focuses on these three issues and proposes a novel estimation of distribution algorithm (EDA) for efficient hyperparameters optimization, with three major contributions in the algorithm design. First, a hybrid-model EDA is proposed to efficiently deal with the mixed-variable difficulty. The proposed algorithm uses a mixed-variable encoding scheme to encode the mixed-variable hyperparameters and adopts an adaptive hybrid-model learning (AHL) strategy to efficiently optimize the mixed-variables. Second, an orthogonal initialization (OI) strategy is proposed to efficiently deal with the challenge of large-scale search space. Third, a surrogate-assisted multi-level evaluation (SME) method is proposed to reduce the expensive computational cost. Based on the above, the proposed algorithm is named s urrogate-assisted hybrid-model EDA (SHEDA). For experimental studies, the proposed SHEDA is verified on widely used classification benchmark problems, and is compared with various state-of-the-art methods. Moreover, a case study on aortic dissection (AD) diagnosis is carried out to evaluate its performance. Experimental results show that the proposed SHEDA is very effective and efficient for hyperparameters optimization, which can find a satisfactory hyperparameters configuration for the CIFAR10, CIFAR100, and AD diagnosis with only 0.58, 0.97, and 1.18 GPU days, respectively.