Iterative Approximation Method Research Articles

Optimal survey schemes for stochastic gradient descent with applications to M-estimation

Iterative stochastic approximation methods are widely used to solve M-estimation problems, in the context of predictive learning in particular. In certain situations that shall be undoubtedly more and more common in the Big Data era, the datasets available are so massive that computing statistics over the full sample is hardly feasible, if not unfeasible. A natural and popular approach to gradient descent in this context consists in substituting the “full data” statistics with their counterparts based on subsamples picked at random of manageable size. It is the main purpose of this paper to investigate the impact of survey sampling with unequal inclusion probabilities on stochastic gradient descent-based M-estimation methods. Precisely, we prove that, in presence of some a priori information, one may significantly increase statistical accuracy in terms of limit variance, when choosing appropriate first order inclusion probabilities. These results are described by asymptotic theorems and are also supported by illustrative numerical experiments.

Joint Spatial Division and Diversity for Massive MIMO Systems

We propose a downlink beamforming scheme that combines spatial division and orthogonal space-time block coding (OSTBC) in multi-user massive MIMO systems. The beamformer is divided into two parts: a pre-beamforming matrix to separate the users into different beams with no interference between each other, which is designed based on the low rank covariance matrix of the downlink channel, and a linear precoding matrix using partial or even no channel state information (CSI) concatenated by an OSTBC. To construct the pre-beamforming matrix, a simple method that selects columns from DFT matrix is presented. To design the linear precoding matrix with partial CSI of the effective channel after the pre-beamforming, we solve an optimization problem to minimize the pairwise error probability (PEP) of the users under an individual power or sum power constraint, respectively. For the individual power constraint, a semi-definite relaxing (SDR) method with a sufficient condition achieving the globally optimal solution is proposed to provide a performance benchmark. In addition, an efficient iterative successive convex approximation (SCA) method is provided to achieve a suboptimal solution. Furthermore, closed form solutions are derived under some special cases. For the sum power constraint, we consider two different designs, i.e., minimizing the average PEP and minimizing the maximum PEP of all users. We find that both non-convex problems have a similar structure, and proposed a unified SCA-Alternating Direction Method of Multipliers (ADMM) algorithm to handle them. The SCA-ADMM method can be implemented in a parallel manner, and thus is with great efficiency. Simulation results show the efficiency of our proposed JSDD scheme and the optimization method.

Efficient Throughput Analysis of Production Lines Based on Modular Queues

Analyzing throughput is very important for the design, optimization, and management of production lines. In this paper, we present an efficient throughput analysis approach for production lines with merging, splitting, and recirculating topologies. In particular, we define a decomposition regulation based on queue modules for the decomposition of the various topologies, and we improve an iterative approximation method for calculating and iterating the queue module state probabilities until the throughput of the production line is obtained. To decrease the computation time and automate the calculation process of the queue module state probabilities, we build a queue module database that includes the solution equations of common queue module state probabilities. The numerical examples show that our approach calculates the throughput of production lines with merging, splitting, and recirculating topologies with high accuracy (≥90%) and efficiency (completed in ten min). Our contribution is an efficient throughput evaluation methodology that can be used to rapidly estimate the performance and the cost of production lines with various topologies in the conceptual design phase of production lines in the industry.

Stability Analysis of Genetic Regulatory Networks With General Random Disturbances.

This paper establishes the stability criteria for genetic regulatory networks with random disturbances. We assume the nonlinear feedback regulation function to satisfy the sector-like condition and the random perturbation to have a finite second-order moment. First, under the globally Lipschitz condition, the existence and uniqueness of solution to random genetic regulatory networks are considered by exploiting an iterative approximation method. Then, by feat of the random analysis method and matrix technique, sufficient conditions are given to guarantee the noise-to-state stability in mean and globally asymptotic stability in probability, respectively. At last, two simulation examples are exploited in order to verify the validity of the proposed theory.

Smoothing Rotary Axes Movements for Ball-End Milling Based on the Gradient-Based Differential Evolution Method

Ball-end milling is widely used in five-axis high-speed machining. The abrupt change of tool orientations or rotary axes movements will scrap the workpiece. This research presents a smoothing method of rotary axes movements within the feasible domains of the rotary-axes space. Most existing smoothing methods of tool orientation or rotary axes movements employ the Dijkstra's shortest path algorithm. However, this algorithm requires extensive computations if the number of the cutter locations is large or the sampling resolution in the feasible regions is high. Moreover, jumps in the results obtained with the Dijkstra's shortest path algorithm may occur, because the optimization problem has to be converted from a continuous problem into a discrete problem when using this algorithm. The progressive iterative approximation (PIA) method incorporating smoothness terms is established as a gradient-based optimization method to smooth the rotary axes movements in this research. Then a gradient-based differential evolution (DE) algorithm, combining the global exploration feature of the DE algorithm and the local searching ability of the gradient-based optimization method, is developed to solve the smoothing model. The validity and effectiveness of the proposed approach are confirmed by numerical examples.

Kinematic analysis and dimensional synthesis of a three-degrees-of-freedom hybrid-drive parallel mechanism

With the increasing demand for the attitude adjustment and vibration isolation of the precision instruments, this paper proposes a three-degrees-of-freedom hybrid-drive parallel mechanism and carries out the kinematic analysis and dimensional synthesis for it. First, the advantages of the hybrid-drive unit are introduced by comparison with the common driving methods. Then, the degrees of freedom of the hybrid-drive parallel mechanism is analyzed by constrained screw method, and the displacement mapping relationship between the generalized input and the moving platform is obtained by a successive approximation iteration method. Furthermore, the first-order and the second-order influence coefficients of the hybrid-drive mechanism are established on the basis of screw theory and the direct method. Besides, the operational dexterity index, the transfer performance index and the stiffness performance index of the hybrid-drive mechanism are introduced, and each single-objective dimensional synthesis of the hybrid-drive mechanism is carried out by the interior point method. Moreover, the multi-objective dimensional synthesis of the hybrid-drive mechanism is proposed by combining the genetic algorithm and the Pareto-optimal solution. Finally, taking the integer numbers of No.23 Pareto set as the numerical example, the kinematics formulas of the hybrid-drive mechanism are verified by the simulation and the experiment.

High-quality point sampling for B-spline fitting of parametric curves with feature recognition

Determining a sequence of points from parametric curves is essential for the construction of least-squares B-spline fitting curves in geometric modeling and related applications. Uniform sampling methods always determine points uniformly in arc length, curvature or a weighted form, but feature points intuitively indicating the curve profile are generally ignored. In this paper, we focus on high-quality point sampling so as to better capture the original curve shape. Firstly, all the feature points (inflection points and extreme curvature points) on the curve are recognized by a parabolic interpolation method. Then more auxiliary points are adaptively added according to the characteristic function defined as a weighted sum of arc length and total curvature characteristics. By adjusting the weight in the characteristic function, it provides more flexibility and controllability to obtain auxiliary points. Finally, we convert the original curve into a B-spline curve by interpolating these feature points and auxiliary points using the progressive–iterative approximation method. Numerical examples demonstrate the effectiveness and high quality of our proposed method.

Approximate iteration detection and precoding in massive MIMO

Massive multiple-input multiple-output provides improved energy efficiency and spectral efficiency in 5G. However it requires large-scale matrix computation with tremendous complexity, especially for data detection and precoding. Recently, many detection and precoding methods were proposed using approximate iteration methods, which meet the demand of precision with low complexity. In this paper, we compare these approximate iteration methods in precision and complexity, and then improve these methods with iteration refinement at the cost of little complexity and no extra hardware resource. By derivation, our proposal is a combination of three approximate iteration methods in essence and provides remarkable precision improvement on desired vectors. The results show that our proposal provides 27%–83% normalized mean-squared error improvement of the detection symbol vector and precoding symbol vector. Moreover, we find the bit-error rate is mainly controlled by soft-input soft-output Viterbi decoding when using approximate iteration methods. Further, only considering the effect on soft-input soft-output Viterbi decoding, the simulation results show that using a rough estimation for the filter matrix of minimum mean square error detection to calculating log-likelihood ratio could provide enough good bit-error rate performance, especially when the ratio of base station antennas number and the users number is not too large.

Iterative computational approach to the solution of the Hamilton-Jacobi-Bellman-Isaacs equation in nonlinear optimal control

In this paper, iterative or successive approximation methods for the Hamilton-Jacobi-Bellman-Isaacs equations (HJBIEs) arising in both deterministic and stochastic optimal control for affine nonlinear systems are developed. Convergence of the methods are established under fairly mild assumptions, and examples are solved to demonstrate the effectiveness of the methods. However, the results presented in the paper are preliminary, and do not yet imply in anyway that the solutions computed will be stabilizing. More improvements and experimentation will be required before a satisfactory algorithm is developed.

A robust approximation method for nonlinear cases of structural reliability analysis

The Hasofer–Lind and Rackwitz–Fiessler (HL-RF) method is a popular iterative approximation method in structural reliability analysis. However, it may pose numerical instability and result in divergence in the face of high nonlinearity. In the present paper, two adjusting parameters are included in this method and a generalization of HL-RF is proposed. The represented parameters are to control the convergence of the sequence especially when nonlinearity increases. The proposed algorithm actually improves the performance of HL-RF in convergence, while it remains as simple as HL-RF to implement without any need to merit functions or line search processes, needed in many approximation methods. Through various numerical examples, the robustness and efficiency of the proposed algorithm in highly nonlinear cases have been shown.

Inexact hierarchical scale separation: A two-scale approach for linear systems from discontinuous Galerkin discretizations

Hierarchical scale separation (HSS) is an iterative two-scale approximation method for large sparse systems of linear equations arising from discontinuous Galerkin (DG) discretizations. HSS splits the linear system into a coarse-scale system of reduced size corresponding to the local mean values of the solution, and a set of decoupled local fine-scale systems corresponding to the higher-order solution components. This scheme then alternates between coarse-scale and fine-scale system solves until both components converge. The motivation of HSS is to promote parallelism by decoupling the fine-scale systems, and to reduce the communication overhead from classical linear solvers by only applying them to the coarse-scale system.We propose a modified HSS scheme (“inexact HSS”, “IHSS”) that exploits the highly parallel fine-scale solver more extensively and only approximates the coarse-scale solution in every iteration, thus resulting in a significant speedup. The tolerance of the coarse-scale solver is adapted in every IHSS cycle, controlled by the residual norm of the fine-scale system. Anderson acceleration is employed in the repeated solving of the fine-scale system to stabilize the scheme. We investigate the applicability of IHSS to systems stemming from the nonsymmetric interior penalty DG discretization of the Cahn–Hilliard equation, discuss its hybrid parallel implementation for large-scale simulations, and compare the performance of a widely used iterative solver with and without IHSS.

Traub-Gander’s family for the simultaneous determination of multiple zeros of polynomials

By combining Traub-Gander’s family of third order for finding a multiple zero and suitable corrective approximations od Schröder’s and Halley’s type, a new family of iterative methods for the simultaneous approximation of multiple zeros of algebraic polynomials is proposed. Taking various forms of a function involved in the iterative formula, a number of different simultaneous methods can be obtained. It is proved that the order of convergence is 4, 5 or 6, depending of the type of employed corrective approximations. Two numerical examples are given to demonstrate the convergence properties of the proposed family of simultaneous methods. Displayed trajectories of the sequences of approximations point to global characteristics of the proposed family of iterative methods.

Generalized B-splines’ geometric iterative fitting method with mutually different weights

In this paper, a novel geometric iterative fitting method is presented which has a set of mutually different weights. It possesses the advantages of least square progressive iterative approximation method (abbr. LSPIA method) which can handle point sets of large sizes and adjust the number of control points and knot vector flexibly. The presenting method degrades into LSPIA method with appropriate choices of weights, and it illustrates better effects for the previous iteration steps comparing with the LSPIA method. Also, this method is further applied to generalized B-splines which have changing core functions (the mentioned generalized B-spline is a special generalization of classical B-spline with linear core function). Combining the advantages of generalized B-splines and choice of different weights, it can handle much more complicated practical problems. Detailed discussion about the choosing of core functions and weights is also given. Plentiful numerical examples are also presented to show the effectiveness of the method.

A qualitative study of a nanotube model using an iterative Taylor method

Physical properties of graphene nanotubes may strongly depend on external fields. In a recent paper V. Jakubský, S. Kuru, J. Negro, J. Phys. A: Math. Theor. 47, 115307 (2014), the authors have studied a model of carbon nanotubes under the presence of an external magnetic field, chosen for some symmetry properties. The model admits an exact solution, provided that the value of a parameter, here denoted as [Formula: see text], be equal to zero. This parameter is the eigenvalue of the component of the momentum in the direction of the nanotube axis. However, it seems that this parameter cannot be discarded for physical reasons. The choice of nontrivial values for this parameter produces an equation of motion for electrons in the nanotube (a Dirac–Weyl equation), which cannot be exactly solvable. Then, we proposed some iterative approximate methods to solve this equation and obtaining its eigenvalues. Some tests have shown that an iterative Taylor method is more efficient than some others we have used. For [Formula: see text], we have found that, excluding the minimal energy eigenvalue, the lowest energy values obtained for [Formula: see text] split into two different ones and, therefore, producing gaps in the energy spectrum.

An iterative computational scheme for solving the coupled Hamilton–Jacobi–Isaacs equations in nonzero-sum differential games of affine nonlinear systems

In this paper, we present iterative or successive approximation methods for solving the coupled Hamilton–Jacobi–Isaacs equations (HJIEs) arising in nonzero-sum differential game for affine nonlinear systems. We particularly consider the ones arising in mixed \({\mathcal H}_{2}/{\mathcal H}_{\infty }\) control. However, the approach is perfectly general and can be applied to any others including those arising in the N-player case. The convergence of the method is established under fairly mild assumptions, and examples are solved to demonstrate the utility of the method. The results are also specialized to the coupled algebraic Riccati equations arising typically in mixed \({\mathcal H}_{2}/{\mathcal H}_{\infty }\) linear control. In this case, a bound within which the optimal solution lies is established. Finally, based on the iterative approach developed, a local existence result for the solution of the coupled-HJIEs is also established.

Spectral partitioning with blends of eigenvectors

Many common methods for data analysis rely on linear algebra. We provide new results connecting data analysis error to numerical accuracy, which leads to the first meaningful stopping criterion for two way spectral partitioning. More generally, we provide pointwise convergence guarantees so that blends (linear combinations) of eigenvectors can be employed to solve data analysis problems with confidence in their accuracy. We demonstrate this theory on an accessible model problem, the Ring of Cliques, by deriving the relevant eigenpairs and comparing the predicted results to numerical solutions. These results bridge the gap between linear algebra based data analysis methods and the convergence theory of iterative approximation methods.

Accuracy of Coupled Cluster Excitation Energies in Diffuse Basis Sets.

We present a comprehensive statistical analysis on the accuracy of various excited state Coupled Cluster methods, accentuating the effect of diffuse basis sets on vertical excitation energies of valence and Rydberg-type states. Many popular approximate doubles and triples methods are benchmarked with basis sets up to aug-cc-pVTZ, with high level EOM-CCSDT results used as reference. The results reveal a serious deficiency of CC2 linear response and CIS(D) techniques in the description of Rydberg states, a feature not shown by the EOM-CCSD(2) and EOM-CCSD variants. The CC3 theory proves to be an accurate choice among the iterative approximate triples methods, while the novel perturbation-based CCSD(T)(a)* variant turns out to be the best way to include the effect of triple excitations in a noniterative way.

Multi-Period Mean-Absolute Deviation Fuzzy Portfolio Selection Model with Entropy Constraints

AbstractThis paper considers a multi-period fuzzy portfolio selection problem maximizing the terminal wealth imposed by risk control, in which the returns of assets are characterized by fuzzy numbers. A fuzzy absolute deviation is originally defined as the risk control of portfolio. Entropy constraints and borrowing constraints are added in the portfolio selection model. Based on the theories of possibility measures, a new multi-period portfolio optimization model with transaction costs is proposed. And then, the proposed model is transformed into a crisp nonlinear programming problem by using fuzzy programming approach. Because of the transaction costs, the multi-period portfolio selection is the dynamic optimization problem with path dependence. Through changing the cost function into a variable, the multi-period portfolio selection is approximately turned into the dynamic programming. Furthermore, the discrete approximate iteration method is designed to obtain the optimal portfolio strategy. Finally, an example is given to illustrate the behavior of the proposed model and the designed algorithm using real data from the Shanghai Stock Exchange.

Multiperiod mean semi-absolute deviation interval portfolio selection with entropy constraints

In this paper, we discuss the uncertain portfolio selection problem where the asset returns are represented by interval data. Since the parameters are interval values, the gain of returns is interval value as well. A new multiperiod mean semi-absolute deviation interval portfolio selection model with the transaction costs, borrowing constraints, threshold constraints and diversification degree of portfolio has been proposed, where the return and risk are characterized by the interval mean and interval semi-absolute deviation of return, respectively. The diversification degree of portfolio is measured by the presented possibilistic entropy. Threshold constraints limit the amount of capital to be invested in each stock and prevent very small investments in any stock. Based on interval theories, the model is converted to a dynamic optimization problem. Because of the transaction costs, the model is a dynamic optimization problem with path dependence. The discrete approximate iteration method is designed to obtain the optimal portfolio strategy. Finally, the comparison analysis of differently desired number of assets and different preference coefficients are provided by numerical examples to illustrate the efficiency of the proposed approach and the designed algorithm.

Integrating capacity analysis with high-speed railway timetabling: A minimum cycle time calculation model with flexible overtaking constraints and intelligent enumeration

Compared with most optimization methods for capacity evaluation, integrating capacity analysis with timetabling can reveal the types of train line plans and operating rules that have a positive influence on improving capacity utilization as well as yielding more accurate analyses. For most capacity analyses and cyclic timetabling methods, the cycle time is a constant (e.g., one or two hours). In this paper, we propose a minimum cycle time calculation (MCTC) model based on the periodic event scheduling problem (PESP) for a given train line plan, which is promising for macroscopic train timetabling and capacity analysis. In accordance with train operating rules, a non-collision constraint and a series of flexible overtaking constraints (FOCs) are constructed based on variations of the original binary variables in the PESP. Because of the complexity of the PESP, an iterative approximation (IA) method for integration with the CPLEX solver is proposed. Finally, two hypothetical cases are considered to analyze railway capacity, and several influencing factors are studied, including train regularity, train speed, line plan specifications (train stops), overtaking and train heterogeneity. The MCTC model and IA method are used to test a real-world case involving the timetable of the Beijing–Shanghai high-speed railway in China.