Efficient Recursive Algorithm Research Articles

An overview on evolving systems and learning from stream data

Evolving systems unfolds from the interaction and cooperation between systems with adaptive structures, and recursive methods of machine learning. They construct models and derive decision patterns from stream data produced by dynamically changing environments. Different components that assemble the system structure can be chosen, being rules, trees, neurons, and nodes of graphs amongst the most prominent. Evolving systems relate mainly with time-varying environments, and processing of nonstationary data using computationally efficient recursive algorithms. They are particularly appropriate for online, real-time applications, and dynamically changing situations or operating conditions. This paper gives an overview of evolving systems with focus on system components, learning algorithms, and application examples. The purpose is to introduce the main ideas and some state-of-the-art methods of the area as well as to guide the reader to the essential literature, main methodological frameworks, and their foundations.

Buffer allocation in asynchronous serial production systems with Bernoulli machines during transients

Some asynchronous production systems can never be truly balanced. This phenomenon demands the incorporation of asynchronism and unreliability in buffer allocation problems (BAPs). This paper reports a BAP in an asynchronous serial production system with Bernoulli machines. A profit-based integer programming model is formulated to maximise the throughput and minimise the buffer capacity simultaneously. Because the production run is finite in the current mass customisation environment, a transient analysis is used to estimate the system performance. Piecewise closed-form expressions are derived for a two-machine case based on Markovian structures. For M > 2 machine lines, an efficient recursive algorithm based on aggregation is developed. Considering the nonlinear characteristic of this BAP, a dynamic-objective particle swarm optimisation with neighbourhood searching is developed. This approach is verified by a case from an assembly plant. The results demonstrate that the programming method can increase the system profit by 1.85%.

Distinguishing node propagation influence by expected index of finite step propagation range

On-line social networks have gradually become an indispensable tool for people. Identifying nodes with high influence in the network as an initial source of communication is of great significance in social perception and rumor control. According to the independent cascade model, in this paper we present an index describing the finite step propagation range expectation as the degree of propagation, and design an efficient recursive algorithm. Based on the local topology information, the index combines the propagation probability to characterize the influence, which can better reflect the propagation influence of a single node. For a single propagation source influence ordering problem, the node degree of propagation and propagation capability are better consistent with each other. And the propagation degree can well describe the propagation influence of nodes under different networks and propagation probabilities. For maximizing the multi-propagation source influence, in this paper we propose a propagation-based heuristic algorithm which is called propagation discount algorithm. This algorithm makes the joint influence of multiple propagation sources maximized. Finally, in this paper we apply the above method to three real networks, showing better effects than the classic indicators and methods. The algorithm has three advantages. First, the expected value of the final propagation range of each node in the small network can be accurately calculated. Second, the degree of propagation fully considers the local topology of the node and belongs to a locality indicator. Third, the indicator combines the effect of propagation probability and yields good outcomes under different networks and propagation probabilities.

Transit timetable synchronization for transfer time minimization

This paper aims to synchronize timetables in a transit network so as to minimize the total passenger transfer waiting time. Assuming a fixed headway for each line, we first formulate the problem as an optimization problem with congruence constraints. We show that the problem is NP-hard, and investigate several special cases of the problem that are solvable in polynomial time. Furthermore, we show that the local search for the general problem is equivalent to the well-studied maximum directed cut problem. As such, we use an approximation algorithm for the maximum directed cut problem to solve the timetable synchronization problem. We assess the quality of the algorithm on a real-world case study, and show that our algorithm significantly outperforms the state-of-the-art in the literature. Lastly, we relax the fixed headway assumption, and propose an efficient recursive quasi-linear time algorithm to minimize the total transfer waiting time in this general setting.

Distributed Krein space-based attack detection over sensor networks under deception attacks

This paper is concerned with distributed attack detection for a discrete time-varying system monitored by a sensor network. An adversary simultaneously launches distinct data deception attacks on both the system dynamics and sensor intercommunication links so as to intentionally falsify the system state and the exchanged sensor measurement outputs. First, delicate distributed estimators are constructed, aiming to provide resilient local estimates for unavailable system state and appropriate residuals for attack detection. Second, an auxiliary Krein space state–space model as well as innovation analysis and a projection technique is skillfully employed to cast the finite horizon distributed estimator design problem into a minimization problem of a certain indefinite quadratic form. A necessary and sufficient condition on the existence of the minimum is derived. Third, a computational efficient recursive algorithm is developed to design desired distributed estimators such that the local estimates and residuals can be both determined at each time step. Furthermore, a two-stage distributed detection mechanism is proposed for each estimator to alert the attack occurrence. Specifically, it is shown that by properly choosing a weighting matrix parameter, each estimator can respectively detect the deception attacks launched on the system layer and sensor intercommunication links. Finally, the effectiveness of the proposed approach is demonstrated through numerical verification.

DCD-Based Recursive Adaptive Algorithms Robust Against Impulsive Noise

The dichotomous coordinate descent (DCD) algorithm has been successfully used for significant reduction in the complexity of recursive least squares (RLS) algorithms. In this brief, we generalize the application of the DCD algorithm to RLS adaptive filtering in impulsive noise scenarios and derive a unified update formula. By employing different robust strategies against impulsive noise, we develop novel computationally efficient DCD-based robust recursive algorithms. Furthermore, to equip the proposed algorithms with the ability to track abrupt changes in unknown systems, a simple variable forgetting factor mechanism is also developed. Simulation results for channel identification scenarios in impulsive noise demonstrate the effectiveness of the proposed algorithms.

Continuous-variable Quantum Phase Estimation based on Machine Learning

Making use of the general physical model of the Mach-Zehnder interferometer with photon loss which is a fundamental physical issue, we investigate the continuous-variable quantum phase estimation based on machine learning approach, and an efficient recursive Bayesian estimation algorithm for Gaussian states phase estimation has been proposed. With the proposed algorithm, the performance of the phase estimation may be improved distinguishably. For example, the physical limits (i.e., the standard quantum limit and Heisenberg limit) for the phase estimation precision may be reached in more efficient ways especially in the situation of the prior information being employed, the range for the estimated phase parameter can be extended from [0, π/2] to [0, 2π] compared with the conventional approach, and influences of the photon losses on the output parameter estimation precision may be suppressed dramatically in terms of saturating the lossy bound. In addition, the proposed algorithm can be extended to the time-variable or multi-parameter estimation framework.

RecurJac: An Efficient Recursive Algorithm for Bounding Jacobian Matrix of Neural Networks and Its Applications

The Jacobian matrix (or the gradient for single-output networks) is directly related to many important properties of neural networks, such as the function landscape, stationary points, (local) Lipschitz constants and robustness to adversarial attacks. In this paper, we propose a recursive algorithm, RecurJac, to compute both upper and lower bounds for each element in the Jacobian matrix of a neural network with respect to network’s input, and the network can contain a wide range of activation functions. As a byproduct, we can efficiently obtain a (local) Lipschitz constant, which plays a crucial role in neural network robustness verification, as well as the training stability of GANs. Experiments show that (local) Lipschitz constants produced by our method is of better quality than previous approaches, thus providing better robustness verification results. Our algorithm has polynomial time complexity, and its computation time is reasonable even for relatively large networks. Additionally, we use our bounds of Jacobian matrix to characterize the landscape of the neural network, for example, to determine whether there exist stationary points in a local neighborhood.

The all configuration mean energy multiconfiguration self-consistent-field method. I. Equal configuration weights

ABSTRACTThe All Configuration Mean Energy (ACME) conditions are a special case of state averaging for Multiconfigurational Self-Consistent-Field (MCSCF) orbital optimisation. The method is formulated using the Graphical Unitary Group Approach (GUGA) in which the Configuration State Function (CSF) basis is represented as walks within a Shavitt graph. This graphical formulation leads to efficient recursive algorithms for the energy and reduced density matrices (RDM) that are independent of the CSF dimension and that scale only as O(n2) where n is the number of occupied orbitals. The Hamiltonian matrix diagonalization step is obviated and the CSF expansion coefficients are neither referenced nor required during the orbital optimisation. This allows MCSCF orbital optimisation to be performed for essentially unlimited numbers of active orbitals and arbitrarily large CSF expansions. The discussion includes various types of CSF expansion spaces, the partitioning of the essential and redundant orbital optimisation parameters, the computation of the spin-density, and the formulation of state-specific analytic gradients and nonadiabatic coupling for high-level electronic structure methods that use the ACME MCSCF orbitals.

Exact analysis of (R, s, S) inventory control systems with lost sales and zero lead time

AbstractWe study an (R, s, S) inventory control policy with stochastic demand, lost sales, zero lead‐time and a target service level to be satisfied. The system is modeled as a discrete time Markov chain for which we present a novel approach to derive exact closed‐form solutions for the limiting distribution of the on‐hand inventory level at the end of a review period, given the reorder level (s) and order‐up‐to level (S). We then establish a relationship between the limiting distributions for adjacent values of the reorder point that is used in an efficient recursive algorithm to determine the optimal parameter values of the (R, s, S) replenishment policy. The algorithm is easy to implement and entails less effort than solving the steady‐state equations for the corresponding Markov model. Point‐of‐use hospital inventory systems share the essential characteristics of the inventory system we model, and a case study using real data from such a system shows that with our approach, optimal policies with significant savings in inventory management effort are easily obtained for a large family of items.

Efficient recursive least-squares algorithms for the identification of bilinear forms

Due to its fast convergence rate, the recursive least-squares (RLS) algorithm is very popular in many applications of adaptive filtering, including system identification scenarios. However, the computational complexity of this algorithm represents a major limitation in applications that involve long filters. Moreover, when the parameter space becomes large, the system identification problem is more challenging and the adaptive filters should be able to cope with this aspect. In this paper, we focus on the identification of bilinear forms, where the bilinear term is defined with respect to the impulse responses of a spatiotemporal model. From this perspective, the solution requires a multidimensional adaptive filtering technique. Recently, the RLS algorithm tailored for bilinear forms (namely RLS-BF) was developed for this purpose. In this framework, the contribution of this paper is mainly twofold. First, in order to reduce the computational complexity of the RLS-BF algorithm, two versions based on the dichotomous coordinate descent (DCD) method are proposed; due to its arithmetic features, the DCD algorithm represents one of the most attractive alternatives to solve the normal equations. However, in the bilinear context, we need to consider the particular structure of the input data and the additional related challenges. Second, in order to improve the robustness of the RLS-BF algorithm in noisy environments, a regularized version is developed, together with a method to find the regularization parameters, which are related to the signal-to-noise ratio (SNR). Furthermore, using a proper estimation of the SNR, a variable-regularized RLS-BF algorithm is designed and two DCD-based low-complexity versions are proposed. Due to their nature, these variable-regularized algorithms have good robustness features against additive noise, which make them behave well in different noisy condition scenarios. Simulation results indicate the good performance of the proposed low-complexity RLS-BF algorithms, with appealing features for practical implementations.

Phylogenetic Comparative Methods on Phylogenetic Networks with Reticulations.

The goal of phylogenetic comparative methods (PCMs) is to study the distribution of quantitative traits among related species. The observed traits are often seen as the result of a Brownian Motion (BM) along the branches of a phylogenetic tree. Reticulation events such as hybridization, gene flow or horizontal gene transfer, can substantially affect a species' traits, but are not modeled by a tree. Phylogenetic networks have been designed to represent reticulate evolution. As they become available for downstream analyses, new models of trait evolution are needed, applicable to networks. We develop here an efficient recursive algorithm to compute the phylogenetic variance matrix of a trait on a network, in only one preorder traversal of the network. We then extend the standard PCM tools to this new framework, including phylogenetic regression with covariates (or phylogenetic ANOVA), ancestral trait reconstruction, and Pagel's $\lambda$ test of phylogenetic signal. The trait of a hybrid is sometimes outside of the range of its two parents, for instance because of hybrid vigor or hybrid depression. These two phenomena are rather commonly observed in present-day hybrids. Transgressive evolution can be modeled as a shift in the trait value following a reticulation point. We develop a general framework to handle such shifts and take advantage of the phylogenetic regression view of the problem to design statistical tests for ancestral transgressive evolution in the evolutionary history of a group of species. We study the power of these tests in several scenarios and show that recent events have indeed the strongest impact on the trait distribution of present-day taxa. We apply those methods to a data set of Xiphophorus fishes, to confirm and complete previous analysis in this group. All the methods developed here are available in the Julia package PhyloNetworks.

A Modified Precise Integration Method for Long-Time Duration Dynamic Analysis

This paper presents a modified Precise Integration Method (PIM) for long-time duration dynamic analysis. The fundamental solution and loading operator matrices in the first time substep are numerically computed employing a known unconditionally stable method (referred to as original method in this paper). By using efficient recursive algorithms to evaluate these matrices in the first time-step, the same numerical results as those using the original method with small time-step are obtained. The proposed method avoids the need of matrix inversion and numerical quadrature formulae, while the particular solution obtained has the same accuracy as that of the homogeneous solution. Through setting a proper value of the time substep, satisfactory accuracy and numerical dissipation can be achieved.

Series Solutions of the Non-Stationary Heun Equation

We consider the non-stationary Heun equation, also known as quantum Painlev\'e VI, which has appeared in different works on quantum integrable models and conformal field theory. We use a generalized kernel function identity to transform the problem to solve this equation into a differential-difference equation which, as we show, can be solved by efficient recursive algorithms. We thus obtain series representations of solutions which provide elliptic generalizations of the Jacobi polynomials. These series reproduce, in a limiting case, a perturbative solution of the Heun equation due to Takemura, but our method is different in that we expand in non-conventional basis functions that allow us to obtain explicit formulas to all orders; in particular, for special parameter values, our series reduce to a single term.

Constant Velocity 3D Convolution

We propose a novel 3-D convolution method, cv3dconv, for extracting spatiotemporal features from videos. It reduces the number of sum-of-products operations in 3-D convolution by thousands of times by assuming the constant moving velocity of the features. We observed that a specific class of video sequences, such as video captured by an in-vehicle camera, can be well approximated with piece-wise linear movements of 2-D features in a temporal dimension. Our principal finding is that a 3-D kernel, represented by constant velocity, can be decomposed into a convolution of a 2-D-shaped kernel and a 3-D-velocity kernel, which is parameterized using only two parameters. We derived an efficient recursive algorithm for this class of 3-D convolution, which is exceptionally suited for sparse spatiotemporal data, and this parameterized decomposed representation imposes a structured regularization along a temporal direction. We experimentally verified the validity of our approximation using a controlled dataset, and we also showed the effectiveness of the cv3dconv by adopting it for deep neural networks (DNNs) in visual odometry estimation task using publicly available event-based camera dataset captured in urban road scene. Our DNN architecture improves the estimation accuracy for about 30% compared with the existing states-of-the-arts architecture designed for event data.

Screw and Lie group theory in multibody dynamics

Screw and Lie group theory allows for user-friendly modeling of multibody systems (MBS), and at the same they give rise to computationally efficient recursive algorithms. The inherent frame invariance of such formulations allows to use arbitrary reference frames within the kinematics modeling (rather than obeying modeling conventions such as the Denavit–Hartenberg convention) and to avoid introduction of joint frames. The computational efficiency is owed to a representation of twists, accelerations, and wrenches that minimizes the computational effort. This can be directly carried over to dynamics formulations. In this paper, recursive O ( n ) Newton–Euler algorithms are derived for the four most frequently used representations of twists, and their specific features are discussed. These formulations are related to the corresponding algorithms that were presented in the literature. Two forms of MBS motion equations are derived in closed form using the Lie group formulation: the so-called Euler–Jourdain or “projection” equations, of which Kane’s equations are a special case, and the Lagrange equations. The recursive kinematics formulations are readily extended to higher orders in order to compute derivatives of the motions equations. To this end, recursive formulations for the acceleration and jerk are derived. It is briefly discussed how this can be employed for derivation of the linearized motion equations and their time derivatives. The geometric modeling allows for direct application of Lie group integration methods, which is briefly discussed.

A sparse partitioned-regression model for nonlinear system–environment interactions

ABSTRACTThis article focuses on the modeling of nonlinear interactions between the design and operational variables of a system and the multivariate outside environment in predicting the system's performance. We propose a Sparse Partitioned-Regression (SPR) model that automatically searches for a partition of the environmental variables and fits a sparse regression within each subdivision of the partition, in order to fulfill an optimal criterion. Two optimal criteria are proposed, a penalized and a held-out criterion. We study the theoretical properties of SPR by deriving oracle inequalities to quantify the risks of the penalized and held-out criteria in both prediction and classification problems. An efficient recursive partition algorithm is developed for model estimation. Extensive simulation experiments are conducted to demonstrate the better performance of SPR compared with competing methods. Finally, we present an application of using building design and operational variables, outdoor environmental variables, and their interactions to predict energy consumption based on the Department of Energy's EnergyPlus data sets. SPR produces a high level of prediction accuracy. The result of the application also provides insights into the design, operation, and management of energy-efficient buildings.

Load Prediction and Distributed Optimal Control of On-Board Battery Systems for Dual-Source Trolleybuses

Dual-source electric vehicles powered by both on-board battery and grid electricity offer unique advantages in fuel economy, cost reduction, driveability, and grid support, which are especially appealing for public transportation in populated cities. The structures of their power supply systems create naturally a networked system of vehicles sharing common feeders. Their power management and control strategies must individually enhance each vehicle’s local performance, and coordinate globally to limit the grid peak current, reduce current fluctuations, and improve efficiency. This paper introduces a novel methodology that employs load prediction, optimal control, and distributed predictive control for current management in such networked systems without vehicle-to-vehicle communications. Estimation and control strategies are introduced, and computationally efficient recursive algorithms are developed. The power system configuration of the Beijing dual-source trolleybus system is used for simulation case studies on the new management strategies. Estimation accuracy, prediction reliability, and performance improvement from the integrated predictive control strategies are demonstrated. Successful implementation of the methodology can potentially attenuate feeder current fluctuations, reduce feeder peak loads, and alleviate disturbances to main grids.

Resource allocation problems in decentralized energy management

Changes in our electricity supply chain are causing a paradigm shift from centralized control towards decentralized energy management. Within the framework of decentralized energy management, devices that offer flexibility in their load profile play an important role. These devices schedule their flexible load profile based on steering signals received from centralized controllers. The problem of finding optimal device schedules based on the received steering signals falls into the framework of resource allocation problems. We study an extension of the traditional problems studied within resource allocation and prove that a divide-and-conquer strategy gives an optimal solution for the considered extension. This leads to an efficient recursive algorithm, with quadratic complexity in the practically relevant case of quadratic objective functions. Furthermore, we study discrete variants of two problems common in decentralized energy management. We show that these problems are NP-hard and formulate natural relaxations of both considered discrete problems that we solve efficiently. Finally, we show that the solutions to the natural relaxations closely resemble solutions to the original, hard problems.

Imaging diffractors using wave-equation migration

We have developed a fast and practical wave-equation-based migration method to image subsurface diffractors. The method is composed of three steps in our implementation. First, it decomposes extrapolated receiver wavefields at every imaging point into local plane waves by a linear Radon transform; the transform is realized by a novel computationally efficient recursive algorithm. Second, the decomposed plane waves are zero lag-correlated with the incident source wavefields, where the incident angles are computed via the structure tensor approach. The resulting prestack images are binned into dip-angle gathers according to the directions of the decomposed plane waves and the calculated incident angles. Third, a windowed median filter is applied to the dip-angle gathers to suppress the focused reflection energy, and it produces the desired diffraction images. This method is tested on synthetic and field data. The results demonstrate that it is resistant to random noise, computationally efficient, and applicable to field data in practice. The results also indicate that the diffraction images are able to provide important discontinuous geologic features, such as scattering and faulting zones, and thus are helpful for seismic interpretation.