Gradient Search Technique Research Articles

Using Recurrent Neural Networks for the Performance Analysis and Optimization of Stochastic Milkrun-Supplied Flow Lines

Long-term throughput, as a key performance indicator of a stochastic flow line, is affected by numerous parameters describing the features of the flow line, such as processing time and buffer size. Fast and accurate evaluation methods for a given set of values for those parameters are a prerequisite to systematically optimize such a flow line. In this paper, we consider the case of a flow line with random processing times, limited buffer capacities and so-called milkruns that supply the machines with material parts that are required to perform, e.g., assembly operations on workpieces. In such a system, shortages in the supply of material parts can limit the performance of the flow line. Up to now, there are no accurate analytical approaches to quantify the complex interactions in such milkrun-supplied flow lines for realistic problem sizes. We propose to use recurrent neural networks to determine the long-term throughput of such flow lines enabling us to evaluate production systems of flexible size. Our results show that the throughput can be determined accurately and quickly via recurrent neural networks. Furthermore, we use this new evaluation procedure as a building block to optimize this type of flow line using gradient and local search techniques.

Machine-learning-based global optimization of microwave passives with variable-fidelity EM models and response features

Maximizing microwave passive component performance demands precise parameter tuning, particularly as modern circuits grow increasingly intricate. Yet, achieving this often requires a comprehensive approach due to their complex geometries and miniaturized structures. However, the computational burden of optimizing these components via full-wave electromagnetic (EM) simulations is substantial. EM analysis remains crucial for circuit reliability, but the expense of conducting rudimentary EM-driven global optimization by means of popular bio-inspired algorithms is impractical. Similarly, nonlinear system characteristics pose challenges for surrogate-assisted methods. This paper introduces an innovative technique leveraging variable-fidelity EM simulations and response feature technology within a kriging-based machine-learning framework for cost-effective global parameter tuning of microwave passives. The efficiency of this approach stems from performing most operations at the low-fidelity simulation level and regularizing the objective function landscape through the response feature method. The primary prediction tool is a co-kriging surrogate, while a particle swarm optimizer, guided by predicted objective function improvements, handles the search process. Rigorous validation demonstrates the proposed framework's competitive efficacy in design quality and computational cost, typically requiring only sixty high-fidelity EM analyses, juxtaposed with various state-of-the-art benchmark methods. These benchmarks encompass nature-inspired algorithms, gradient search, and machine learning techniques directly interacting with the circuit's frequency characteristics.

POTAM: A Parallel Optimal Task Allocation Mechanism for Large-Scale Delay Sensitive Mobile Edge Computing

Design an optimization model for task management among Mobile Terminal (MT), Macro cell Base Station (MBS), and multiple Small cell Base Stations (SBS) for the large-scale Mobile Edge Computing (MEC) system, is a challenging issue due to the large number of tasks and SBSs. Inspired by this, we propose a Parallel Optimal Task Allocation Mechanism (POTAM) framework for MEC, which includes Device to Device (D2D)-enabled computing, MBS computing and Edge Computation Resource Distribution (ECRD) computing. In POTAM, we exploit a parallel multi-block Alternating Direction Method of Multipliers (ADMM) based method to model both requirements of delay and energy consumptions, which formulates the task allocation under these requirements as a nonlinear 0–1 integer programming problem. To solve this problem, we develop an efficient combination of conjugate gradient, Newton and linear search techniques based algorithm with Logarithmic Smoothing and Cyclic Block coordinate Gradient Projection (CBGP) methods, which can guarantee convergence and reduce computational complexity with a good scalability. In order to allocate task cooperatively, an optimal approach is proposed, ECRD-A, which is used to find the shortest path among each node. Numerical results demonstrate the effectiveness of the POTAM and it can effectively reduce delay and energy consumption for a large-scale MEC system.

Nonlinear model identification of dissimilar laser joining of S.S 304 and ABS using the Hammerstein–Wiener method

The nonlinear system identification Hammerstein–Wiener method is used according to the experimental outputs of laser joining of Acrylonitrile Butadiene Styrene (ABS) and stainless steel 304 concerning the temperature distribution in different parameters of a pulsed laser, including welding speed, focal length, pulse duration, frequency, power and current. Limited linear and nonlinear models exist for such processes; however, due to restrictions and complex numerical calculations, their use in the real-time process control is not practical. An accurate model for the dissimilar laser joining of metals and plastic materials is essential for controller design and process automation. Compared to other popular nonlinear compensation techniques, this model has a simpler structure and needs fewer calculations. This paper considers the identification problems of Hammerstein–Wiener nonlinear systems by employing the gradient search technique. Seven available data sets for training and test phases in dissimilar input parameters were used, and root-mean-square error (RMSE), mean absolute percentage error (MAPE) and fitness value were calculated. The results indicated that the identified Hammerstein-Wiener model had acceptable performance in training and test phases and was in good agreement with experimental results.

Two‐stage auxiliary model gradient‐based iterative algorithm for the input nonlinear controlled autoregressive system with variable‐gain nonlinearity

SummaryThis article focuses on the parameter estimation problem of the input nonlinear system where an input variable‐gain nonlinear block is followed by a linear controlled autoregressive subsystem. The variable‐gain nonlinearity is described analytical by using an appropriate switching function. According to the gradient search technique and the auxiliary model identification idea, an auxiliary model‐based stochastic gradient algorithm with a forgetting factor is presented. For the sake of improving the parameter estimation accuracy, an auxiliary model gradient‐based iterative algorithm is proposed by utilizing the iterative identification theory. To further optimize the performance of the algorithm, we decompose the identification model of the system into two submodels and derive a two‐stage auxiliary model gradient‐based iterative (2S‐AM‐GI) algorithm by using the hierarchical identification principle. The simulation results confirm the effectiveness of the proposed algorithms and show that the 2S‐AM‐GI algorithm has higher identification efficiency compared with the other two algorithms.

Using Joint Generalized Eigenvectors of a Set of Covariance Matrix Pencils for Deflationary Blind Source Extraction

In this paper, we develop a new deflationary blind source extraction (BSE) algorithm that extracts source signals in a sequential fashion via the joint generalized eigenvectors of a set of covariance matrix pencils. The new concept of joint generalized eigenvector is defined. We prove that these vectors can be made unique and identical to the source extraction vectors with properly selected matrix pencils. To resolve the open problem of estimating joint generalized eigenvectors, we develop an approach based on the deflation operation and the proportional property of the joint generalized eigenvectors. Specifically, with the proportional property, we show that the estimation problem can be formulated as an optimization involving a quadratic cost function and a unit-rank matrix constraint. An efficient iterative algorithm is then developed by applying the gradient search, matrix shrinkage, deflation, and symmetry-preserving vectorization techniques. This algorithm estimates the joint generalized eigenvectors and conducts BSE sequentially. Its computational complexity and convergence are analyzed. Simulations demonstrate that this algorithm outperforms many typical BSE or blind source separation algorithms. In particular, the new algorithm is more robust to both heavy noise and ill-conditioned mixing matrices.

Cost Optimal Design of a Power Inductor by Sequential Gradient Search

Power inductors are used for compensating VAR generated by long EHV transmission lines and in electronic circuits. For the EHV-lines, the rating of the inductor is decided upon by techno-economic considerations on the basis of the line-susceptance. It is a high voltage high current device, absorbing little active power and large reactive power. The cost is quite high- hence the design should be made cost-optimally. The 3-phase power inductor is similar in construction to a 3-phase core-type transformer with the exception that it has only one winding per phase and each limb is provided with an air-gap, the length of which is decided upon by the inductance required. In this paper, a design methodology based on sequential gradient search technique and the corresponding algorithm leading to cost-optimal design of a 3-phase EHV power inductor has been presented. The case-study has been made on a 220 kV long line of NHPC running from Chukha HPS to Birpara of Coochbihar.

Gradient Search Technique Applied to Dual-Optimal Design of a 3-Phase Core Type Transformer

Most of the power and distribution transformers in power system are 3-phase core type transformers. They are extensively used- as such their cost is a sizable proportion of the total system cost. Therefore they are to be designed cost-optimally. The optimality is with reference to an objective function, generally in presence of constraints. The design variables are to be varied within their given bounds and the optimal solution is to be reached in finite number of steps, satisfying the given constraints. The paper presents a method for optimizing the design, in presence of constraints specified by the customer and the regulatory authorities, through gradient search technique. The objective function is a weighted combination of the cost of the transformer and the running losses which is not a mathematically framed but obtained from the transformer design sub-routine through computer programme. The objective of this paper is to find out above objective function without mathematically framed function which leads to inaccuracy for approximation due to magnetic saturation and nonlinearity present in the system.Most of the objective function is framed by taking few terms from a polynomial to avoid complicacy of the function, such as core loss expression obtained from curve with the variation of flux densities. The starting point has been chosen within the allowable parameter space- the steepest decent path has been followed for convergence. The step length has been judiciously chosen. The hyper surface has been found to be concave. As such no local minima problem is faced. The method is best as it its convergence is quickest.

Simulation and sensitivity analysis for heavy linear paraffins production in LAB production Plant

Abstract Linear alkyl benzene (LAB) is vastly utilized for the production of biodegradable detergents and emulsifiers. Predistillation unit is a part of LAB production plant in which that produced heavy linear paraffins (nC10-nC13). In this study, a mathematical model has been developed for heavy linear paraffins production in distillation columns, which has been solved using a commercial code. The models have been validated by the actual data. The effects of process parameters such as reflux rate, and reflux temperature using Gradient Search technique has been investigated. The sensitivity analysis shows that optimum reflux in columns are achieved.

Modeling and optimization of local constraint elastomer treatments for vibration and noise reduction

This paper presents the vibroacoustic study of a constrained elastomer treatment used in the industry for reducing noise. It can be trimmed and bonded conveniently to vibrating structures for reducing radiated noise. First, an identification of the elastomer viscoelastic characteristics is carried out with a program that models damped vibrations, a conjugate gradient search technique and experimental data extracted rom two contact-free modal analyses. The first modal analysis, adapted to dissipation characterization, is made on a partially covered suspended plate. The second modal analysis, adapted to identifying the elastomer stiffness behavior, concerns a cantilever beam that has almost been covered by a large treatment. The complete dynamic characterization is finally deduced from an iterative procedure that combines information from both experiments. The procedure highlights the influence of the treatment bonding quality on the achieved elastomer damping. Second, practical rules are deduced from a number of parametric studies on beams with baffled radiation conditions. In particular, a design criterion is introduced to help positioning patches where the elastomer damping can be maximized. A threshold, for which an optimal acoustic performance with a minimum of elastomer can be fulfilled, is also identified.

Cost-Optimal Design of a 3-Phase Core Type Transformer by Gradient Search Technique

3-phase core type transformers are extensively used as power and distribution transformers in power system and their cost is a sizable proportion of the total system cost. Therefore they should be designed cost-optimally. The design methodology for reaching cost-optimality has been discussed in details by authors like Ramamoorty. It has also been discussed in brief in some of the text-books of electrical design. The paper gives a method for optimizing design, in presence of constraints specified by the customer and the regulatory authorities, through gradient search technique. The starting point has been chosen within the allowable parameter space the steepest decent path has been followed for convergence. The step length has been judiciously chosen and the program has been maneuvered to avoid local minimal points. The method appears to be best as its convergence is quickest amongst different optimizing techniques.

Variational method for untangling and optimization of spatial meshes

A variational method that can provably construct 3D quasi-isometric mappings between domains of a complex shape is introduced. A local maximum principle for polyconvex mesh element distortion measures is formulated. It allows us to control the invertibility and distortion bounds for non-simplicial elements in the minimization process. A simple and efficient technique for construction of boundary orthogonal meshes suggested in Garanzha (2000) is applied to the construction of hexahedral meshes and thick prismatic mesh layers around complex shapes. The mesh untangling technique, which is a generalization of the penalty method suggested in Garanzha and Kaporin (1999), is verified on a wide set of challenging test problems. Another untangling technique based on theoretical ideas from Ivanenko (1997) is implemented and tested. It provably constructs admissible meshes using a finite number of minimization steps. A minimization technique for the mesh distortion functional is described. The approach is based on the global gradient search technique with preconditioning and domain decomposition for local mesh optimization and untangling. Application areas for explicit and implicit minimization methods are evaluated.

Stock Market Prediction Using Artificial Neural Networks

In this study we apply back propagation Neural Network models to predict the daily Shanghai Stock Exchange Composite Index. The learning algorithm and gradient search technique are constructed in the models. We evaluate the prediction models and conclude that the Shanghai Stock Exchange Composite Index is predictable in the short term. Empirical study shows that the Neural Network models is successfully applied to predict the daily highest, lowest, and closing value of the Shanghai Stock Exchange Composite Index, but it can not predict the return rate of the Shanghai Stock Exchange Composite Index in short terms.

Optimization of Continuous Post-Tensioned Concrete Bridge Girders of Non-Uniform Depth

Post-tensioned single-cell concrete box girders of variable depth are used in continuous bridges to achieve both economy and aesthetics. In this paper, the minimum prestressing for a given two- or three-span continuous bridge deck profile is achieved through a constrained optimization procedure using linear programming in conjunction with gradient method of optimization. The constrained nonlinear problem is solved by transforming it to a linear one by introducing a new design variable which proportions long (full length) and short tendons. A gradient search technique is used to solve iteratively the optimization problem. The design variables include interior to exterior span ratio (for three-span), the depth profile of the cross-section, tendon profile, and prestressing force. The total cost of the member is taken as the cost of structural materials (concrete and prestressing steel), excluding the cost of formwork. The design constraints are the prescribed limits on the allowable stresses, ultimate strength, and serviceability requirements as per ACI 343R-95. A purpose-built computer program has been developed to readily determine the optimum prestressing for a given bridge deck profile. From solutions of a number of hypothetical bridge girders using AASHTO H20 bridge loading, several useful results related to design are obtained. Results show that for a symmetrical three-span and two-span continuous bridge girder, the optimum prestressing consists of an optimum arrangement of both long and short tendons, and for a three-span girder of prescribed length, the optimum design is attained at an optimum ratio of interior to exterior span with 1.35 to 1.45.

Slow sampling on-line harmonics/interharmonics estimation technique for smart meters

This paper presents a modified gradient search (MGS) technique to estimate harmonics/interharmonics of power system voltages and currents. The proposed technique, which is applied to spectral estimation, consumes very low processing power and needs a few samples per cycle for real-time implementation in smart meters. The computational burden of the proposed technique is lower than the discrete Fourier transform (DFT) and fast Fourier transform (FFT) and can be implemented recursively on digital signal processors (DSPs). In addition, the effects of slow sampling on the accuracy and estimation latency have been investigated. Performance of the proposed method is evaluated by simulations in MATLAB-Simulink and confirmed by experimental results.

An Approximation Approach for Identification of Stochastic Max-Plus Linear Systems

AbstractMax-plus linear systems belong to a special class of discrete-event systems that consists of systems with synchronization but no choice. Our focus in this paper is on stochastic max-plus linear systems, i.e., perturbed systems that are linear in the max-plus algebra. One interesting topic is the identification of such systems. Previous works report on a method for identifying the parameters of a state space model for a stochastic max-plus linear system from measured data. However, due to the structure of such systems, this method results in a complex identification problem. Therefore, the aim of this paper is to decrease the computational complexity and the computation time of this problem. To this end, we use an approximation approach that is based on the higher-order raw moments of a random variable. This method results in a less complex problem that can be solved efficiently using gradient search techniques.

A projected–gradient interior–point algorithm for complementarity problems

Interior---point algorithms are among the most efficient techniques for solving complementarity problems. In this paper, a procedure for globalizing interior---point algorithms by using the maximum stepsize is introduced. The algorithm combines exact or inexact interior---point and projected---gradient search techniques and employs a line---search procedure for the natural merit function associated with the complementarity problem. For linear problems, the maximum stepsize is shown to be acceptable if the Newton interior---point search direction is employed. Complementarity and optimization problems are discussed, for which the algorithm is able to process by either finding a solution or showing that no solution exists. A modification of the algorithm for dealing with infeasible linear complementarity problems is introduced which, in practice, employs only interior---point search directions. Computational experiments on the solution of complementarity problems and convex programming problems by the new algorithm are included.

Artificial neural network: border detection in echocardiography

Being non-invasive and low cost, the echocardiography has become a diagnostic technique largely applied for the determination of the left ventricle systolic and diastolic volumes, which are used indirectly to calculate the left ventricle ejection volume, the cardiac cavities muscular contraction, the regional ejection fraction, the myocardial thickness, and the ventricular mass, etc. However, the image is very noisy, which renders the delineation of the borders of the left ventricle very difficult. While there are many techniques image segmentation, this work chooses the artificial neural network (ANN) since it is not very sensitive to noise. In order to reduce the processing time, the operator selects the region of interest where the neural network will identify the borders. Neighborhood and gradient search techniques are then employed to link the points and the left ventricle contour is traced. The present method has been efficient in detecting the left ventricle borders echocardiography images compared to those whose borders were delineated by the specialists. For good results, it is important to choose properly the areas to be analyzed and the central points of these areas.

Evaluating the process of a genetic algorithm to improve the back-propagation network: A Monte Carlo study

Many studies have mapped a bit-string genotype using a genetic algorithm to represent network architectures to improve performance of back-propagation networks (BPN). But the limitations of gradient search techniques applied to complex nonlinear optimization problems have often resulted in inconsistent and unpredictable performance. This study focuses on how to collect and re-evaluate the weight matrices of a BPN while the genetic algorithm operations are processing in each generation to optimize the weight matrices. In this way, overfitting, a drawback of BPNs that usually occurs during the later stage of neural network training with descending training error and ascending prediction error, can also be avoided. This study extends the parameters and topology of the neural network to enhance the feasibility of the solution space for complex nonlinear problems. The value of the proposed model is compared with previous studies using a Monte Carlo study on in-sample, interpolation, and extrapolation data for six test functions.

A hybrid optimization method of evolutionary and gradient search

This article proposes a hybrid optimization algorithm, which combines evolutionary algorithms (EA) and the gradient search technique, for optimization with continuous parameters. Inheriting the advantages of the two approaches, the new method is fast and capable of global search. The main structure of the new method is similar to that of EA except that a special individual called the gradient individual is introduced and EA individuals are located symmetrically. The gradient individual is propagated through generations by means of the quasi-Newton method. Gradient information required for the quasi-Newton method is calculated from the costs of EA individuals produced by the evolution strategies (ES). The symmetric placement of the individuals with respect to the best individual is for calculating the gradient vector by the central difference method. For the estimation of the inverse Hessian matrix, symmetric Rank-1 update shows better performance than BFGS and DFP. Numerical tests on various benchmark problems and a practical control design example demonstrate that the new hybrid algorithm gives a faster convergence rate than EA, without sacrificing the capability of global search.