Stochastic Optimization Algorithm Research Articles

A new numerical method for processing longitudinal data: clinical applications


 Background: Processing longitudinal data is a computational issue that arises in many applications, such as in aircraft design, medicine, optimal control and weather forecasting. Given some longitudinal data, i.e. scattered measurements, the aim consists in approximating the parameters involved in the dynamics of the considered process. For this problem, a large variety of well-known methods have already been developed.
 Results: Here, we propose an alternative approach to be used as effective and accurate tool for the parameters fitting and prediction of individual trajectories from sparse longitudinal data. In particular, our mixed model, that uses Radial Basis Functions (RBFs) combined with Stochastic Optimization Algorithms (SOMs), is here presented and tested on clinical data. Further, we also carry out comparisons with other methods that are widely used in this framework.
 Conclusions: The main advantages of the proposed method are the flexibility with respect to the datasets, meaning that it is effective also for truly irregularly distributed data, and its ability to extract reliable information on the evolution of the dynamics.

Method of adjustment of three circuit system of active shielding of magnetic field in multi-storey buildings from overhead power lines with wires triangular arrangement

Aim. For the first time the method of adjustment of three circuit system of the active shielding of the magnetic field based on experimentally determined space-time characteristics to increase the shielding factor in a multi-storey building located near a single-circuit overhead transmission lines with a wires triangular arrangement was developed. Methodology. When synthesizing the laboratory model of system of active shielding the coordinates of spatial arrangement and of three shielding coils, the currents in shielding coils and resulting magnetic flux density value in the shielding space were calculated. The synthesis is based on the multi-criteria game decision, in which the payoff vector is calculated on the basis on quasi-stationary approximation solutions of the Maxwell equations. The game decision is calculated based on the stochastic particles multi swarm optimization algorithms. Results. Computer simulation and experimental research of the space-time characteristics of laboratory model of three circuit system of active shielding of magnetic field, generated by overhead power lines with phase conductors triangle arrangements in multi-storey building are given. The possibility of initial magnetic flux density level reducing to the sanitary standards level is shown. Originality. For the first time the synthesis and adjustment of laboratory model of three circuit system of active shielding of magnetic field based on experimentally determined space-time characteristics to increase the shielding factor in a multi-storey building located near a single-circuit overhead transmission lines with a wires triangular arrangement carried out. Practical value. Practical recommendations from the point of view of the implementation of developed method of adjustment of the three circuit system of the active shielding of the magnetic field based on experimentally determined space-time characteristics in a multi-storey building located near a single-circuit overhead transmission lines with a triangular arrangement of wires are given.

An Active Set Limited Memory BFGS Algorithm for Machine Learning

In this paper, a stochastic quasi-Newton algorithm for nonconvex stochastic optimization is presented. It is derived from a classical modified BFGS formula. The update formula can be extended to the framework of limited memory scheme. Numerical experiments on some problems in machine learning are given. The results show that the proposed algorithm has great prospects.

Texture Optimization and Verification for the Thrust Bearing Used in Rotary Compressors Based on a Transient Tribo-Dynamics Model

Abstract Rotary compressors are designed more and more compact, and the compressor cylinder’s ambient pressure is designed very high to facilitate oil separation and improve efficiency. However, these designs cause the working condition of the thrust bearing becoming harsher, and severe wear may occur. The present study is aimed at mitigating its wear condition through surface texturing. Based on a transient tribo-dynamics model considering the coupling effect of the journal and thrust bearings, a texture optimization study for the thrust bearing is conducted, in which three different stochastic optimization algorithms are utilized. The results show that thrust bearings with optimized textures have significantly reduced contact forces and wear under a high working frequency due to an extra hydrodynamic support around the texture dimples. The optimized texture designs are fabricated on the thrust bearing surfaces by a high-accurate picosecond laser machine, and their performance is assessed through experiments using a compressor performance test platform. The experiment results confirm that the textured thrust bearing has a lower wear depth. Moreover, the coefficient of performance (COP) of the testing compressor with textured thrust bearing is increased while its input power decreases, which implies a reduced friction force and a higher energy efficiency.

PopDMMO: A general framework of population-based stochastic search algorithms for dynamic multimodal optimization

Dynamic multimodal optimization problems (DMMOPs) are a class of problems consisting of two characteristics, i.e., dynamic and multimodal natures. The former characteristic reveals that the properties of DMMOPs change over time, which is derived from dynamic optimization problems (DOPs). The latter one indicates that there exist multiple global or acceptable local optima, which comes from the multimodal optimization problems (MMOPs). Although there has been much attention to both DOPs and MMOPs in the field of meta-heuristics, there is little work devoting to the connection between the dynamic and multimodal natures in DMMOPs. To solve DMMOPs, the strategies dealing with dynamic and multimodal natures in the algorithms should cooperate with each other. Before looking deeply into the connections between two natures, there is necessary to measure the performances of the methods dealing with two natures in DMMOPs. In this paper, first, considering the dynamic and multimodal natures of DMMOPs, we design a set of benchmark problems to simulate various dynamic and multimodal environments. Then, we propose the optimization framework called PopDMMO containing several popular algorithms and methods to test and compare the performances of these algorithms, which gives a general view of solving DMMOPs.

A Novel Ferroelectric FET-Based Adaptively-Stochastic Neuron for Stimulated-Annealing Based Optimizer With Ultra-Low Hardware Cost

In this work, based on ferroelectric FET (FeFET), a novel capacitor-less and bio-inspired adaptively-stochastic neuron is proposed and experimentally demonstrated for the first time for solving optimization problems in spiking neural network with remarkably reduced hardware cost. By exploiting the physics of inherent stochastic dynamic process of ferroelectric domains nucleation, the proposed FeFET-based neuron with only three transistors can realize adaptively stochastic spike firing behavior, where its stochasticity gradually decreases during operation. The adaptive stochasticity is experimentally demonstrated for the hardware implementation of stochastic simulated annealing algorithm for optimization, providing a promising ultralow-hardware-cost solution for solving optimization problems.

Pelican Optimization Algorithm: A Novel Nature-Inspired Algorithm for Engineering Applications.

Optimization is an important and fundamental challenge to solve optimization problems in different scientific disciplines. In this paper, a new stochastic nature-inspired optimization algorithm called Pelican Optimization Algorithm (POA) is introduced. The main idea in designing the proposed POA is simulation of the natural behavior of pelicans during hunting. In POA, search agents are pelicans that search for food sources. The mathematical model of the POA is presented for use in solving optimization issues. The performance of POA is evaluated on twenty-three objective functions of different unimodal and multimodal types. The optimization results of unimodal functions show the high exploitation ability of POA to approach the optimal solution while the optimization results of multimodal functions indicate the high ability of POA exploration to find the main optimal area of the search space. Moreover, four engineering design issues are employed for estimating the efficacy of the POA in optimizing real-world applications. The findings of POA are compared with eight well-known metaheuristic algorithms to assess its competence in optimization. The simulation results and their analysis show that POA has a better and more competitive performance via striking a proportional balance between exploration and exploitation compared to eight competitor algorithms in providing optimal solutions for optimization problems.

Superquantile-Based Learning: A Direct Approach Using Gradient-Based Optimization

We consider a formulation of supervised learning that endows models with robustness to distributional shifts from training to testing. The formulation hinges upon the superquantile risk measure, also known as the conditional value-at-risk, which has shown promise in recent applications of machine learning and signal processing. We show that, thanks to a direct smoothing of the superquantile function, a superquantile-based learning objective is amenable to gradient-based optimization, using batch optimization algorithms such as gradient descent or quasi-Newton algorithms, or using stochastic optimization algorithms such as stochastic gradient algorithms. A companion software SPQR implements in Python the algorithms described and allows practitioners to experiment with superquantile-based supervised learning.

Training Multi-Layer Perceptron with Enhanced Brain Storm Optimization Metaheuristics

In the domain of artificial neural networks, the learning process represents one of the most challenging tasks. Since the classification accuracy highly depends on the weights and biases, it is crucial to find its optimal or suboptimal values for the problem at hand. However, to a very large search space, it is very difficult to find the proper values of connection weights and biases. Employing traditional optimization algorithms for this issue leads to slow convergence and it is prone to get stuck in the local optima. Most commonly, back-propagation is used for multi-layer-perceptron training and it can lead to vanishing gradient issue. As an alternative approach, stochastic optimization algorithms, such as nature-inspired metaheuristics are more reliable for complex optimization tax, such as finding the proper values of weights and biases for neural network training. In this work, we propose an enhanced brain storm optimization-based algorithm for training neural networks. In the simulations, ten binary classification benchmark datasets with different difficulty levels are used to evaluate the efficiency of the proposed enhanced brain storm optimization algorithm. The results show that the proposed approach is very promising in this domain and it achieved better results than other state-of-the-art approaches on the majority of datasets in terms of classification accuracy and convergence speed, due to the capability of balancing the intensification and diversification and avoiding the local minima. The proposed approach obtained the best accuracy on eight out of ten observed dataset, outperforming all other algorithms by 1–2% on average. When mean accuracy is observed, the proposed algorithm dominated on nine out of ten datasets.

Correlating Rheological Experiments, TEVP Models, and Microstructure with Small Angle Light Scattering

The existence of an evolving microstructure in a 2.9 vol% fumed silica in paraffin oil and polyisobutylene is demonstrated experimentally and via rheological modeling during steady state and large amplitude oscillatory shear flow. The continuously evolving, rebuilding, and breaking down of the microstructure is shown, and correlated through the rheology experiments, thixo-elastovisco-plastic modeling, and small angle light scattering (SALS). All elements are then connected via a global, stochastic optimization algorithm that will provide parameter estimation with a “best-fit” of the steady state and transient data using the well-known Modified Delaware Thixotropic Model, allowing for the comparison of SALS results with experimental values.

A Decentralized Stochastic Algorithm for Coupled Composite Optimization With Linear Convergence

A Decentralized Stochastic Algorithm for Coupled Composite Optimization With Linear Convergence

Zeroth and First Order Stochastic Frank-Wolfe Algorithms for Constrained Optimization

This paper considers stochastic convex optimization problems with two sets of constraints: (a) deterministic constraints on the domain of the optimization variable, which are difficult to project onto; and (b) deterministic or stochastic constraints that admit efficient projection. Problems of this form arise frequently in the context of semidefinite programming as well as when various NP-hard problems are solved approximately via semidefinite relaxation. Since projection onto the first set of constraints is difficult, it becomes necessary to explore projection-free algorithms, such as the stochastic Frank-Wolfe (FW) algorithm. On the other hand, the second set of constraints cannot be handled in the same way, and must be incorporated as an indicator function within the objective function, thereby complicating the application of FW methods. Similar problems have been studied before; however, they suffer from slow convergence rates. This work, equipped with momentum based gradient tracking technique, guarantees fast convergence rates on par with the best-known rates for problems without the second set of constraints. Zeroth-order variants of the proposed algorithms are also developed and again improve upon the state-of-the-art rate results. We further propose the novel trimmed FW variants that enjoy the same convergence rates as their classical counterparts, but are empirically shown to require significantly fewer calls to the linear minimization oracle speeding up the overall algorithm. The efficacy of the proposed algorithms is tested on relevant applications of sparse matrix estimation, clustering via semidefinite relaxation, and uniform sparsest cut problem.

APPLICATION OF STOCHASTIC PARTICLE SWARM OPTIMIZATION ALGORITHM FOR NONINVASIVE DETERMINATION OF TEMPERATURE-DEPENDENT THERMAL PROPERTIES OF BIOLOGICAL TISSUE

The knowledge of thermal properties of biological tissue is of great importance to medical diagnosis and thermotherapy. In this study, an inverse model based on stochastic particle swarm optimization (SPSO) algorithm and sensitivity analysis is developed to determine the temperature-dependent thermal properties of one-dimensional biological tissue. The noninvasive determination task is formulated as an inverse problem which is solved by the SPSO algorithm from the knowledge of surface temperature measurements. A logistic mapping strategy is introduced to improve the quality of the initial particle swarm. Retrieval results demonstrate that the developed inverse technique is robust to determine the temperature-dependent thermal conductivity, specific heat and blood perfusion rate of biological tissue. The maximum relative error of the simultaneous determination results is less than 0.8%. The maximum determination error is 1.15% when a 3% random deviation is considered. Meanwhile, the present SPSO algorithm is proved to be more efficient and accurate than other intelligent algorithms. Considering the convergence accuracy and computational time, the population size of the SPSO algorithm is suggested to be M = 80.

Robust Speech Emotion Recognition Using CNN+LSTM Based on Stochastic Fractal Search Optimization Algorithm

One of the main challenges facing the current approaches of speech emotion recognition is the lack of a dataset large enough to train the currently available deep learning models properly. Therefore, this paper proposes a new data augmentation algorithm to enrich the speech emotions dataset with more samples through a careful addition of noise fractions. In addition, the hyperparameters of the currently available deep learning models are either handcrafted or adjusted during the training process. However, this approach does not guarantee finding the best settings for these parameters. Therefore, we propose an optimized deep learning model in which the hyperparameters are optimized to find their best settings and thus achieve more recognition results. This deep learning model consists of a convolutional neural network (CNN) composed of four local feature-learning blocks and a long short-term memory (LSTM) layer for learning local and long-term correlations in the log Mel-spectrogram of the input speech samples. To improve the performance of this deep network, the learning rate and label smoothing regularization factor are optimized using the recently emerged stochastic fractal search (SFS)-guided whale optimization algorithm (WOA). The strength of this algorithm is the ability to balance between the exploration and exploitation of the search agents’ positions to guarantee to reach the optimal global solution. To prove the effectiveness of the proposed approach, four speech emotion datasets, namely, IEMOCAP, Emo-DB, RAVDESS, and SAVEE, are incorporated in the conducted experiments. Experimental results confirmed the superiority of the proposed approach when compared with state-of-the-art approaches. Based on the four datasets, the achieved recognition accuracies are 98.13%, 99.76%, 99.47%, and 99.50%, respectively. Moreover, a statistical analysis of the achieved results is provided to emphasize the stability of the proposed approach.

A stochastic topology optimization algorithm for improved fluid dynamics systems

Abstract The use of topology optimization in the design of fluid dynamics systems is still in its infancy. With the decreasing cost of additive manufacture, the application of topology optimization in the design of structural components has begun to increase. This paper provides a method for using topology optimization to reduce the power dissipation of fluid dynamics systems, with the novelty of it being the first application of stochastic mechanisms in the design of 3D fluid–solid geometrical interfaces. The optimization algorithm uses the continuous adjoint method for sensitivity analysis and is optimized against an objective function for fluid power dissipation. The paper details the methodology behind a vanilla gradient descent approach before introducing stochastic behavior through a minibatch-based system. Both algorithms are then applied to a novel case study for an internal combustion engine's piston cooling gallery before the performance of each algorithm's resulting geometry is analyzed and compared. The vanilla gradient descent algorithm achieves an 8.9% improvement in pressure loss through the case study, and this is surpassed by the stochastic descent algorithm which achieved a 9.9% improvement, however this improvement came with a large time cost. Both approaches produced similarly unintuitive geometry solutions to successfully improve the performance of the cooling gallery.

Private Stochastic Dual Averaging for Decentralized Empirical Risk Minimization

In this work, we study the decentralized empirical risk minimization problem under the constraint of differential privacy (DP). Based on the algorithmic framework of dual averaging, we develop a novel decentralized stochastic optimization algorithm to solve the problem. The proposed algorithm features the following: i) it perturbs the stochastic subgradient evaluated over individual data samples, with which the information about the dataset can be released in a differentially private manner; ii) it employs hyperparameters that are more aggressive than conventional decentralized dual averaging algorithms to speed up convergence. The upper bound for the utility loss of the proposed algorithm is proven to be smaller than that of existing methods to achieve the same level of DP. As a by-product, when removing the perturbation, the non-private version of the proposed algorithm attains the optimal O(1/t) convergence rate for non-smooth stochastic optimization. Finally, experimental results are presented to demonstrate the effectiveness of the algorithm.

Quantization Enabled Privacy Protection in Decentralized Stochastic Optimization

By enabling multiple agents to cooperatively solve a global optimization problem in the absence of a central coordinator, decentralized stochastic optimization is gaining increasing attention in areas as diverse as machine learning, control, and sensor networks. Since the associated data usually contain sensitive information, such as user locations and personal identities, privacy protection has emerged as a crucial need in the implementation of decentralized stochastic optimization. In this paper, we propose a decentralized stochastic optimization algorithm that is able to guarantee provable convergence accuracy even in the presence of aggressive quantization errors that are proportional to the amplitude of quantization inputs. The result applies to both convex and non-convex objective functions, and enables us to exploit aggressive quantization schemes to obfuscate shared information, and hence enables privacy protection without losing provable optimization accuracy. In fact, by using a stochastic ternary quantization scheme, which quantizes any value to three numerical levels, we achieve quantization-based rigorous differential privacy in decentralized stochastic optimization, which has not been reported before. In combination with the presented quantization scheme, the proposed algorithm ensures, for the first time, rigorous differential privacy in decentralized stochastic optimization without losing provable convergence accuracy. Simulation results for a distributed estimation problem as well as numerical experiments for decentralized learning on a benchmark machine learning dataset confirm the effectiveness of the proposed approach.

Raw Material Ordering and Transportation Problems in Production Enterprises

This paper analyzes from the perspective of the enterprise, establishes the corresponding objective function based on the requirements of different problems, and solves the optimal solution through the stochastic optimization algorithm combined with the corresponding constraints. Taking suppliers as the research object, this paper constructs the unordered rate index for suppliers from the perspective of enterprises. The average difference between supply and demand and the stability of the difference between supply and demand are adjusted. Finally, an evaluation model is constructed based on the grey comprehensive evaluation method to obtain the ranking of the importance of each supplier in ensuring the production of enterprises. Based on this ranking, the 50 most important suppliers are determined.

Archery Algorithm: A Novel Stochastic Optimization Algorithm for Solving Optimization Problems

Finding a suitable solution to an optimization problem designed in science is a major challenge. Therefore, these must be addressed utilizing proper approaches. Based on a random search space, optimization algorithms can find acceptable solutions to problems. Archery Algorithm (AA) is a new stochastic approach for addressing optimization problems that is discussed in this study. The fundamental idea of developing the suggested AA is to imitate the archer's shooting behavior toward the target panel. The proposed algorithm updates the location of each member of the population in each dimension of the search space by a member randomly marked by the archer. The AA is mathematically described, and its capacity to solve optimization problems is evaluated on twenty-three distinct types of objective functions. Furthermore, the proposed algorithm's performance is compared vs. eight approaches, including teaching-learning based optimization, marine predators algorithm, genetic algorithm, grey wolf optimization, particle swarm optimization, whale optimization algorithm, gravitational search algorithm, and tunicate swarm algorithm. According to the simulation findings, the AA has a good capacity to tackle optimization issues in both unimodal and multimodal scenarios, and it can give adequate quasi-optimal solutions to these problems. The analysis and comparison of competing algorithms’ performance with the proposed algorithm demonstrates the superiority and competitiveness of the AA.

Моделювання переміщення вантажів на основі мурашиного алгоритму за допомогою групи безпілотних літальних апаратів

This article is about issues that happen during moving a large number of similar goods that are located on a certain site and concentrating them in one place (warehouse). These issues include imperfect conditions of facilities, unsatisfactory state of transport service, exhausted rolling stocks, low quality or overloading of transport routs, remote sites of goods’ reception and delivery points, poor managing of goods moving, unexpected outcomes, etc. In order to solve these problems the authors recommend using of a group of unpiloted air vehicles (UAV) and solving the issues related to an efficient managing of their moving by the means of stochastic optimization algorithm, namely the ANTS ant algorithm. Authors propose the improved methods of an ant algorithm ANT that uses the function of changing the intensity of the digital phenomenon, and, unlike existing algorithms, uses not linear, but cubic dynamic scaling of the change of the digital phenomenon that allow us to focus on searching not only the short routes of shipping but also on consideration of a new one. The experiments on goods movement by the different quantities of involved UAV have been conducted using the modeling in WeBots and the test Mavic 2Pro UAVs for shipping 150 g typical loads to the one site (storehouse). The assessment of efficiency of loads shipping was conducted on the base of its results and a relation between a quantity of shipped loads and a time of shipping was defined. It was found that the efficacy of this process increases with increase in the number of UAV because the time of moving decreases. Also, it was found that the every next increase in the number of involved UAVs causes the smaller increase in efficiency due to the waiting in a line for a load’s disembarking.