Gradient Search Technique Research Articles

Perturbation-Based Eigenvector Updates for On-Line Principal Components Analysis and Canonical Correlation Analysis

Principal components analysis is an important and well-studied subject in statistics and signal processing. Several algorithms for solving this problem exist, and could be mostly grouped into one of the following three approaches: adaptation based on Hebbian updates and deflation, optimization of a second order statistical criterion (like reconstruction error or output variance), and fixed point update rules with deflation. In this study, we propose an alternate approach that avoids deflation and gradient-search techniques. The proposed method is an on-line procedure based on recursively updating the eigenvector and eigenvalue matrices with every new sample such that the estimates approximately track their true values as would be calculated analytically from the current sample estimate of the data covariance matrix. The perturbation technique is theoretically shown to be applicable for recursive canonical correlation analysis, as well. The performance of this algorithm is compared with that of a structurally similar matrix perturbation-based method and also with a few other traditional methods like Sanger's rule and APEX.

Modelling and Simulation of Universal Digital AC Bridge Using Ann Technique

Bridges are the measuring devices used for accurate measurement of impedance. As such, there is no universal bridge circuit that measures impedance of all types of passive components. Moreover, the bridges available cannot be used for high-frequency impedance measurements. This paper proposes an automatic digital AC bridge model, which is considered to be universal and suitable for high-frequency impedance measurement up to 1MHz. In this adaptive digital AC Bridge, balancing is achieved by means of Artificial Neural Network based on stochastic gradient search algorithm. Among the various gradient search techniques, the Widrow-Hoff's Least Mean Square (LMS) technique has been chosen as it involves less computational burden. The digital AC bridge operations have been verified by simulation. The simulated system was tested for measuring impedances of pure and impure inductors, capacitors, resistors, and a series combination of an inductance and a capacitance. The effects of various parameters like supply voltage, reference resistance, learning rate, and sampling rate were considered to investigate the performance of universal high-precision AC Bridge, and the results are reported. This model is advantageous over the traditional heuristic methods in terms of higher accuracy, faster convergence, and greater reproducibility and reliability.

Simultaneous optimization of sequential IMRT plans

Radiotherapy often comprises two phases, in which irradiation of a volume at risk for microscopic disease is followed by a sequential dose escalation to a smaller volume either at a higher risk for microscopic disease or containing only gross disease. This technique is difficult to implement with intensity modulated radiotherapy, as the tolerance doses of critical structures must be respected over the sum of the two plans. Techniques that include an integrated boost have been proposed to address this problem. However, clinical experience with such techniques is limited, and many clinicians are uncomfortable prescribing nonconventional fractionation schemes. To solve this problem, we developed an optimization technique that simultaneously generates sequential initial and boost IMRT plans. We have developed an optimization tool that uses a commercial treatment planning system (TPS) and a high level programming language for technical computing. The tool uses the TPS to calculate the dose deposition coefficients (DDCs) for optimization. The DDCs were imported into external software and the treatment ports duplicated to create the boost plan. The initial, boost, and tolerance doses were specified and used to construct cost functions. The initial and boost plans were optimized simultaneously using a gradient search technique. Following optimization, the fluence maps were exported to the TPS for dose calculation. Seven patients treated using sequential techniques were selected from our clinical database. The initial and boost plans used to treat these patients were developed independently of each other by dividing the tolerance doses proportionally between the initial and boost plans and then iteratively optimizing the plans until a summation that met the treatment goals was obtained. We used the simultaneous optimization technique to generate plans that met the original planning goals. The coverage of the initial and boost target volumes in the simultaneously optimized plans was equivalent to the independently optimized plans actually used for treatment. Tolerance doses of the critical structures were respected for the plan sum; however, the dose to critical structures for the individual initial and boost plans was different between the simultaneously optimized and the independently optimized plans. In conclusion, we have demonstrated a method for optimization of initial and boost plans that treat volume reductions using the same dose per fraction. The method is efficient, as it avoids the iterative approach necessitated by currently available TPSs, and is generalizable to more than two treatment phases. Comparison with clinical plans developed independently suggests that current manual techniques for planning sequential treatments may be suboptimal.

Tabu search optimization of horizontal and vertical alignments of forest roads

This article describes a program for optimizing horizontal and vertical alignments of forest roads using Tabu search, a modern heuristic technique. Once a series of intersection points (IPs) is selected manually, the program generates alternative horizontal and vertical alignments. The program precisely generates ground profile and cross sections using a high-resolution digital elevation model (DEM) derived from light detection and ranging (LiDAR) data. It accurately calculates earthwork volumes for curved roadways using the Pappus-based method. The program also estimates construction and maintenance costs. Tabu search optimizes forest road alignments based on the total costs. The application of the program to part of Capitol State Forest in Washington State, USA, indicated that the program successfully found better alignments than manually selected initial alignments. The effect of initial solutions and the number of iterations on the Tabu search process was examined. The result showed that the solutions were improved using the best solutions with the smaller number of grade change points as the initial solutions. It also showed that a small number of iterations could be used to reduce computation time due to the fact that Tabu search is based on a gradient search technique. Finally, the Dijkstra method was examined to find initial solutions without manually initialized solutions. The program, when combined with the Dijkstra method, could find similar-quality solutions from manually initialized solutions. The program will become useful with further tests and verifications.

ON ITERATIVE FEEDBACK TUNING AND DISTURBANCE REJECTION USING SIMPLE NOISE MODELS

The objective of this contribution is to discuss three basic control design methods for disturbance rejection. The key idea is to use a simple fixed continuous time noise model as a design variable to specify the desirable closed loop properties, e.g., the closed loop bandwidth. We will study three closely related controller design methods based on the noise model, namely minimum variance control, weighted minimum variance control and a loop shaping design procedure proposed by Skogestad and Postlethwaite. All these methods lead to simple controller structures, where the transfer function of the input/output relation is the only unknown part.The main motivation of this study has been tuning of controllers for disturbance rejection by means of iterative feedback tuning (IFT). Here, the controller parameters are found by minimizing a specified cost function using gradient search techniques. The gradients are estimated using data from iterative experiments. This is a rather difficult problem if no external input excitation is allowed. Import factors for good results are the controller parametrization and the choice of cost function.We propose an IFT cost function based on a simple output signal weighting tailored to the fixed noise model controller structure. This leads to robust procedure for iterative feedback tuning of a controller for disturbance rejection. The approach is illustrated on a simple example.

Globally convergent algorithms for dc operating point analysis of nonlinear circuits

An important objective in the analysis of an electronic circuit is to find its quiescent or dc operating point. This is the starting point for performing other types of circuit analysis. The most common method for finding the dc operating point of a nonlinear electronic circuit is the Newton-Raphson method (NR), a gradient search technique. There are known convergence issues with this method. NR is sensitive to starting conditions. Hence, it is not globally convergent and can diverge or oscillate between solutions. Furthermore, NR can only find one solution of a set of equations at a time. This paper discusses and evaluates a new approach to dc operating-point analysis based on evolutionary computing. Evolutionary algorithms (EAs) are globally convergent and can find multiple solutions to a problem by using a parallel search. At the operating point(s) of a circuit, the equations describing the current at each node are consistent and the overall error has a minimum value. Therefore, we can use an EA to search the solution space to find these minima. We discuss the development of an analysis tool based on this approach. The principles of computer-aided circuit analysis are briefly discussed, together with the NR method and some of its variants. Various EAs are described. Several such algorithms have been implemented in a full circuit-analysis tool. The performance and accuracy of the EAs are compared with each other and with NR. EAs are shown to be robust and to have an accuracy comparable to that of NR. The performance is, at best, two orders of magnitude worse than NR, although it should be noted that time-consuming setting of initial conditions is avoided.

On the stress-wave imaging of cavities in a semi-infinite solid

The problem of mapping underground cavities from surface seismic measurements is investigated within the framework of a regularized boundary integral equation (BIE) method. With the ground modeled as a uniform elastic half-space, the inverse analysis of elastic waves scattered by a three-dimensional void is formulated as a task of minimizing the misfit between experimental observations and theoretical predictions for an assumed void geometry. For an accurate treatment of the gradient search technique employed to solve the inverse problem, sensitivities of the predictive BIE model with respect to cavity parameters are evaluated semi-analytically using an adjoint problem approach and a continuum kinematics description. Several key features of the formulation, including the rigorous treatment of the radiation condition for semi-infinite solids, modeling of an illuminating seismic wave field, and treatment of the prior information, are highlighted. A set of numerical examples with spherical and ellipsoidal cavity geometries is included to illustrate the performance of the method. It is shown that the featured adjoint problem approach reduces the computational requirements by an order of magnitude relative to conventional finite-difference estimates, thus rendering the three-dimensional elastic-wave imaging of solids tractable for engineering applications.

Short Maturity Options and Jump Memory: An Empirical Analysis

We investigate Memory using an extensive data base of short-term S&P 500 Index options. Jump memory refers to the attenuation of the jump intensity and magnitude parameters following a jump event. Behavioral and rational explanations for this phenomenon are posited. The pricing accuracy of the jump-diffusion model under parameter restrictions is also investigated. A restricted version of the model simplifies estimation, does not appear to degrade pricing accuracy, and permits clearer identification of components of jump memory. Further, a genetic algorithm is shown to be both robust and to produce parameter estimates that significantly improve pricing performance vis a' vis pricing performance when parameters are estimated by a gradient search technique.

Lyapunov-theory-based radial basis function networks for adaptive filtering

Two important convergence properties of Lyapunov-theory-based adaptive filtering (LAF) adaptive filters are first explored. The LAF finite impulse response and infinite impulse response adaptive filters are then realized using the radial basis function (RBF) neural networks (NNs). The proposed adaptive RBF neural filtering system possesses the distinctive properties of RBF NN and the LAF filtering system. Unlike many adaptive filtering schemes using gradient search techniques, a Lyapunov function of the error between the desired signal and the RBF NN output is first defined. By properly choosing the weights update law in the Lyapunov sense, the RBF filter output can asymptotically converge to the desired signal. The design is independent of the stochastic properties of the input disturbances and the stability is guaranteed by the Lyapunov stability theory. Simulation examples for nonlinear adaptive prediction of nonstationary signal and system identification are performed.

Coupled Waveform Analysis in Dynamic Characterization of Lossy Solids

By means of the viscoelastic wave propagation model for a homogeneous semi-infinite solid, a rational analytical and computational framework is developed for nonintrusive, wave-based characterization of lossy media. The problem of estimating the equivalent uniform material properties from surface observations is formulated in the Bayesian setting for two canonical testing configu- rations, one involving a vertically and the other a horizontally polarized wave field. By enhancing the treatment of the inverse problem through a fully coupled viscoelastic analysis of the observed waveforms, the method provides consistent estimates of the in situ material stiffness, damping, and density which are not available from conventional seismic interpretations. For a rigorous treatment of the gradient search technique employed by the inverse solution, the necessary derivatives of the predictive model are evaluated analytically. A set of results with noise-contaminated synthetic measurements is included to highlight several key features of the back analysis.

A genetic classification error method for speech recognition

In this paper, we present a genetic approach for training hidden Markov models using minimum classification error (MCE) as the reestimation criteria. This approach is discriminative and proved to be better than other non-discriminative approach such as the maximum likelihood (ML) method. The major problem of using the MCE is to formulate the error rate estimate as a smooth continuous loss function such that the gradient search techniques can be applied to search for the solutions. A genetic approach for this particular classification error method aimed at finding the global solution or better optimal solutions is proposed. Comparing our approach with the ML and MCE approaches, the experimental results showed that it is superior to both the MCE and ML methods.

An Analytic Stochastic Model for the Transit Vehicle Holding Problem

This paper describes an analytic model that determines the optimal vehicle holding time at a control stop along a transit route. This model is based on a stochastic transit service model presented by Andersson and Scalia-Tomba (1981) and enhanced by Marguier (1985). The use of a stochastic service model allows greater realism in the analytic modeling. Making use of these results, the paper presents an analytic model that may be used to determine the optimal holding time for a vehicle at a control stop. As it is formulated, the single vehicle holding problem is a convex quadratic program in a single variable, and is easily solved using gradient or line search techniques. The analytic holding model overcomes two noted problems in the literature: it includes stochastic service attributes of vehicle running times and passenger boarding and alighting processes, and the model may be used for real-time control purposes. The use and potential benefits of the model are illustrated in a simple example. This model may be useful in developing a computerized decision support system to enhance the effectiveness of transit operational decision-making.

Neural Networks Refined: Using a Genetic Algorithm to Identify Predictors of is Student Success

In this study, a genetic algorithm trained neural network was used to identify relevant characteristics of successful IS students. Those variables identified as predictors of student success included a student's age, gender, and the Myers-Briggs' attributes of extraversion/introversion (EI) and sensing/intuition (SN). During the past decade, neural networks have gained popularity, as they work exceedingly well for mapping unknown functions from historical data. With a neutral network, a researcher can simply include a wide variety of inputs in the model and the neural network will learn to discriminate between the relevant inputs and those that are irrelevant. Since the majority of neural network research uses gradient search techniques, usually some form of backpropagation, there is no way to identify the inputs that actually contribute to the prediction. By using the genetic algorithm as an alternative search technique, these contributing relevant variables can be identified.

Development and study of an automatic AC bridge for impedance measurement

An automatic ac bridge method employing the principle of balancing by means of a stochastic gradient search algorithm is presented. Among the various gradient search techniques, the Widrow-Hoff least mean square (LMS) technique has been chosen as it involves a low computational burden. The LMS algorithm has been used by the authors for the operation of an ac bridge operating with continuous variables. The relevant operation has been verified by simulation; it can also measure negative impedance. A relation has been established between parameters to be used in the LMS algorithm for discrete and continuous versions of the bridge. The balance convergence time for a relatively simple form of the LMS bridge, namely, the R-R type bridge, has been theoretically established and experimentally verified both by simulation and real-time implementation. The frequency response of the LMS algorithm has also been determined.

Comparative evaluation of genetic algorithm and backpropagation for training neural networks

In view of several limitations of gradient search techniques (e.g. backpropagation), global search techniques, including evolutionary programming and genetic algorithms (GAs), have been proposed for training neural networks (NNs). However, the effectiveness, ease-of-use, and efficiency of these global search techniques have not been compared extensively with gradient search techniques. Using five chaotic time series functions, this paper empirically compares a genetic algorithm with backpropagation for training NNs. The chaotic series are interesting because of their similarity to economic and financial series found in financial markets.

A Modified Gradient-Search Training Technique for Neural-Network Structural Control

This paper discusses a neural-network controller used to reduce the seismic response of building structures. The training process is carried out through a novel gradient-search technique that minimizes a desired cost function defined in terms of a set of structural response quantities. The proposed training approach does not require an additional neural network for the identification of the structural system. The training method is quite flexible with regard to the choice of the cost function to be minimized. A series of earthquake, base-acceleration time histories are considered to train the network. The effectiveness of the controller is demonstrated through several sets of numerical results obtained for a multistory building equipped with an active tuned mass damper.

Sheet Thickness Optimization for Superplastic Forming of Engineering Structures

This paper explores two optimization strategies; the gradient search and proportional control methods, for determining the initial sheet thickness of superplastic forming to ensure final desired part thickness. A hemispherical dome model was involved in the testing of both optimization methods. Also, a three-dimensional rectangular box model was optimized by the proportional control method. The gradient search technique is shown to be acceptable in terms of the optimized thickness obtained, but displays poor convergence rates. The proportional control approach presented is easy to be implemented, and yields not only more accurate sheet thickness, but much higher convergence speeds that makes such optimization possible on complex geometric models. [S1087-1357(00)01001-7]

OPTIMUM POST-TENSIONING FOR THREE-SPAN CONTINUOUS SLAB-TYPE BRIDGE DECKS

An optimization procedure is prescribed in this paper to find the minimum prestressing force and the corresponding tendon profile required for three-span continuous slab-type bridge decks using a combination of both long and short tendons. The constrained nonlinear problem is solved by first transforming it into a linear one with the introduction of a new design variable which defines the proportion of long and short tendons. The problem is then solved iteratively by linear programming in conjunction with a gradient search technique. The results show that for a three-span continuous bridge deck with the interior span longer than the exterior one, the optimum prestressing consists of a combination of both long and short tendons. For the three-span bridge of a prescribed length, the procedure determines the optimal ratio of the interior to the exterior span for which the required prestressing force is minimum.

Optimal Induction Motor Design by the Use of NLP-Approximation Approach and SQP-Method

In computational terms, design optimization for any electrical machine constitutes a mixedinteger programming (MIP) task because, along with continuously variable design parameters, which are allowed to assume any value within a specified range, there exist parameters that may only assume discrete values; finding an optimal solution for such a combination of design variables represents a computational challenge. If, on the other hand, for the purpose of searching for the optima, discrete variables could be treated as continuously variable quantities, the task would be considerably simpler. Then a range of modern optimization methods based on gradient search techniques could be employed in determining the search direction. This approach would be tantamount to having converted the MIP problem into a nonlinear programming (NLP) problem. This paper describes the application of a sequential quadratic programming (SQP) technique in design optimization for an induction motor. It demonstrates how a MIP-problem can be successfully approached using an NLP-approximation to simplify the task of finding optima. The design algorithm, implemented on a desktop computer, allows globally optimized designs to be found with relative ease, unlikely to be achievable with conventional design methods. Aspects of algorithm implementation are discussed, including the formulation of the NLP-approximation, convergence speed, and the nature of convergence.

Simulated Annealing 2-D Nonlinear Inversion of Geomagnetic Deep Sounding Data Near Ujjain-Guna, India

—An attempt has been made to use the global optimization technique of Simulated Annealing (S.A.) inversion to interpret the conductivity structure derived from the geomagnetic deep sounding data of N–W India. The present results supersede the earlier result obtained by 2-D conventional linearized inversion. The earlier linearized inversion, following an iterative gradient search technique on the same data set, has been re-evaluated and further constrained through an exhaustive search of the parameter space. The conductive response of an hypothesized conductor located between Ujjain and Guna, India, has been modelled by this random search tool. The location of the proposed model is now in agreement with the theory, since the conductive bodies are centered exactly below the center of the response function. In an earlier attempt by linearized inversion, this was not feasible. The central body is located at a depth of 19 km from the surface, suggesting a thickness of 15 km and resistivity of 14 ohm.m. The resistivity contrast of this ensemble of conductive bodies with the background varies by a factor of 100 to 385. This low conductivity contrast with respect to the background is in conformity to the low temperature as inferred from the available heat-flow data in the region. A marginally different estimate of the conductivity (normally mid-crustal conductors are assigned conductivities of the order of 50 to 200 ohm.m) for the mid-crustal conductor has been found. Existence of a mid-crustal conductor is clearly indicated which was also not detected in the earlier study. Low geo-temperature gradients existing in the region rule out the possibility of a thermal origin for this mid-crustal conductor. A likely explanation could be due to the presence of graphitic carbon at lower crustal depths. However, the role of electrolytic fluid present in the interconnected pore—spaces of rocks may be another tangible explanation.