Gradient Ascent Method Research Articles

Independent component analysis by lp-norm optimization

In this paper, a couple of new algorithms for independent component analysis (ICA) are proposed. In the proposed methods, the independent sources are assumed to follow a predefined distribution of the form f(s)=αexp(−β|s|p) and a maximum likelihood estimation is used to separate the sources. In the first method, a gradient ascent method is used for the maximum likelihood estimation, while in the second, a non-iterative algorithm is proposed based on the relaxation of the problem. The maximization of the log-likelihood of the estimated source XTw given the parameter p and the data X is shown to be equivalent to the minimization of lp-norm of the projected data XTw. This formulation of ICA has a very close relationship with the Lp-PCA where the maximization of the same objective function is solved. The proposed algorithm solves an approximation of the lp-norm minimization problem for both super-(p < 2) and sub-Gaussian (p > 2) cases and shows superior performance in separating independent sources than the state of the art algorithms for ICA computation.

Gradient-driven parking navigation using a continuous information potential field based on wireless sensor network

Wireless sensor networks can support building and transportation system automation in numerous ways. An emerging application is to guide drivers to promptly locate vacant parking spaces in large parking structures during peak hours. This paper proposes efficient parking navigation via a continuous information potential field and gradient ascent method. Our theoretical analysis proves the convergence of a proposed algorithm and efficient convergence during the first and second steps of the algorithm to effectively prevent parking navigation from a gridlock situation. The empirical study demonstrates that the proposed algorithm performs more efficiently than existing algorithms.

Maximum Power Point Tracking of a Wind Power Plant With Predictive Gradient Ascent Method

In this paper, we present maximum power point tracking for a wind power plant (WPP) using the gradient ascent (GA) in a data-driven manner. The conventional GA method achieves fast convergent performance by considering only direct wake terms when calculating the axial induction factors. However, the conventional method might not be close to optimal even when the wind conditions are steady state. In this paper, we propose a new method using the relationships between the direct and indirect wake terms. Using the relationship between the wake terms can prevent sudden deviations after convergence to a single operating point, even when significant indirect wake terms exist in the presence of multiple wakes. Therefore, the proposed method provides not only fast convergence to an operating point, but also closer-to-optimal power production without sudden deviations compared to the conventional method. We validated the effectiveness of the proposed method using modeled WPP layouts with various wind conditions.

Iterative Multiplicative Filters for Data Labeling

Based on an idea in [4] we propose a new iterative multiplicative filtering algorithm for label assignment matrices which can be used for the supervised partitioning of data. Starting with a row-normalized matrix containing the averaged distances between prior features and the observed ones the method assigns in a very efficient way labels to the data. We interpret the algorithm as a gradient ascent method with respect to a certain function on the product manifold of positive numbers followed by a reprojection onto a subset of the probability simplex consisting of vectors whose components are bounded away from zero by a small constant. While such boundedness away from zero is necessary to avoid an arithmetic underflow, our convergence results imply that they are also necessary for theoretical reasons. Numerical examples show that the proposed simple and fast algorithm leads to very good results. In particular we apply the method for the partitioning of manifold-valued images.

A weighted information criterion for multiple minor components and its adaptive extraction algorithms

Minor component (MC) plays an important role in signal processing and data analysis, so it is a valuable work to develop MC extraction algorithms. Based on the concepts of weighted subspace and optimum theory, a weighted information criterion is proposed for searching the optimum solution of a linear neural network. This information criterion exhibits a unique global minimum attained if and only if the state matrix is composed of the desired MCs of an autocorrelation matrix of an input signal. By using gradient ascent method and recursive least square (RLS) method, two algorithms are developed for multiple MCs extraction. The global convergences of the proposed algorithms are also analyzed by the Lyapunov method. The proposed algorithms can extract the multiple MCs in parallel and has advantage in dealing with high dimension matrices. Since the weighted matrix does not require an accurate value, it facilitates the system design of the proposed algorithms for practical applications. The speed and computation advantages of the proposed algorithms are verified through simulations.

Joint Source-Relay Design for Full-Duplex MIMO AF Relay Systems

The performance of full-duplex (FD) relay systems can be greatly impacted by the self-interference (SI) at relays. By exploiting multi-antenna in FD relay systems, the spectral efficiency of FD relay systems can be enhanced through spatial SI mitigation. This paper studies joint source transmit beamforming and relay processing to achieve rate maximization for FD MIMO amplify-and-forward (AF) relay systems with consideration of relay processing delay. The problem is difficult to solve due mainly to the SI constraint induced by the relay processing delay. In this paper, we first present a sufficient condition under which the relay amplification matrix has rank one structure. Then, for the case of rank one amplification matrix, the rate maximization problem is equivalently simplified into an unconstrained problem which can be locally solved using gradient ascent method. Next, we propose a penalty-based algorithmic framework, called P-BSUM, for a class of constrained optimization problems which have difficult equality constraints in addition to some convex constraints. By rewriting the rate maximization problem with a set of auxiliary variables, we apply the P-BSUM algorithm to the rate maximization problem in the general case. Finally, numerical results validate the efficiency of the proposed algorithms and show that the joint source-relay design approach under the rank one assumption could be strictly suboptimal as compared to the P-BSUM-based joint source-relay design approach.

Towards rational catalyst design: a general optimization framework.

Rational catalyst design is one of the most fundamental goals in heterogeneous catalysis. Herein, we briefly review our previous design work, and then introduce a general optimization framework, which converts catalyst design into an optimization problem. Furthermore, an example is given using the gradient ascent method to show how this framework can be used for rational catalyst design. This framework may be applied to other design schemes.

Mixtures of Conditional Random Fields for Improved Structured Output Prediction.

The conditional random field (CRF) is a successful probabilistic model for structured output prediction problems. In this brief, we consider to enlarge the representational capacity of CRF via mixture modeling. The motivation is that a single CRF can perform well if the data conform to the statistical dependence assumption imposed by the CRF model structure, whereas it may potentially fail to model the data that come from multiple different sources or domains. For the conventional conditional likelihood objective, we derive the expectation-maximization algorithm in conjunction with the direct gradient ascent method for learning a CRF mixture with sequence or image-structured data. In addition, we provide alternative mixture learning algorithms that aim to maximize either the classification margin or the sitewise conditional likelihood, which were previously shown to outperform the conventional estimator for single CRF models in a variety of situations. We demonstrate the improved prediction accuracy of the proposed mixture learning algorithms on several important sequence labeling problems.

Clever eye algorithm for target detection of remote sensing imagery

Target detection algorithms for hyperspectral remote sensing imagery, such as the two most commonly used remote sensing detection algorithms, the constrained energy minimization (CEM) and matched filter (MF), can usually be attributed to the inner product between a weight filter (or detector) and a pixel vector. CEM and MF have the same expression except that MF requires data centralization first. However, this difference leads to a difference in the target detection results. That is to say, the selection of the data origin could directly affect the performance of the detector. Therefore, does there exist another data origin other than the zero and mean-vector points for a better target detection performance? This is a very meaningful issue in the field of target detection, but it has not been paid enough attention yet. In this study, we propose a novel objective function by introducing the data origin as another variable, and the solution of the function is corresponding to the data origin with the minimal output energy. The process of finding the optimal solution can be vividly regarded as a clever eye automatically searching the best observing position and direction in the feature space, which corresponds to the largest separation between the target and background. Therefore, this new algorithm is referred to as the clever eye algorithm (CE). Based on the Sherman–Morrison formula and the gradient ascent method, CE could derive the optimal target detection result in terms of energy. Experiments with both synthetic and real hyperspectral data have verified the effectiveness of our method.

Cardiac Arrhythmia Classification Using Atrial Activity Signal

Electrocardiogram (ECG) analysis is the method for cardiac arrhythmia diagnosis. Cardiac arrhythmia is a group of diseases in which the heart beat shows irregularities. The classification of the correct type of Arrhythmia is a necessary and critical issue which becomes difficult in certain cases. In certain cases the Atrial Electrical Activity(AEA) signals are hidden in some other waves and they cannot be detected and classified easily from surface. This paper presents a method by which classification of cardiac arrhythmia is done by processing the ECG. The procedure is done by identifying the QRS complex and thus its position in the preprocessed digitised ECG. The AEA signals in the ECG is identified using a semi-automatic method called Separation Using Maximum Energy Ratio (SUMER), which is found efficient in identifying the hidden waves. In SUMER, an energy ratio based cost function is created and maximized using the gradient ascent method. After identifying the first AEA, the precise positions of all the AEA waves are identified. The ratio between the QRS complex and the AEA waves is calculated and based on the ratio the arrhythmia type is identified. The proposed method would help in better and easy classification of arrhythmia at its early stage.

Naive Bayes vs Logistic Regression: Theory, Implementation and Experimental Validation

This article presents the theoretical derivation as well as practical steps for implementing Naive Bayes (NB) and Logistic Regression (LR) classifiers. A generative learning under Gaussian Naive Bayes assumption and two discriminative learning techniques based on gradient ascent and Newton-Raphson methods are described to estimate the parameters of LR. Some limitation of learning techniques and implementation issues are discussed as well. A set of experiments are performed for both the classifiers under different learning circumstances and their performances are compared. From the experiments, it is observed that LR learning with gradient ascent technique outperforms general NB classifier. However, under Gaussian Naive Bayes assumption, both classifiers NB and LR perform similar.

Energy-Efficient Resource Allocation for Fractional Frequency Reuse in Heterogeneous Networks

Next generation wireless networks face the challenge of increasing energy consumption while satisfying the unprecedented increase in the data rate demand. To address this problem, we propose a utility-based energy-efficient resource allocation algorithm for the downlink transmissions in heterogeneous networks (HetNets). We consider the fractional frequency reuse (FFR) method in order to mitigate the intra- and inter-cell interference. The proposed algorithm divides the resource allocation problem into frequency and power assignment problems and sequentially solves them. The proposed power control algorithm uses the gradient ascent method to control the transmit power of macrocell base stations (MeNBs) as most of the power in the network is consumed there. We present the optimality conditions of the resource allocation problem and the convergence of the proposed algorithm. In order to mitigate the inter-cell interference further, we study the interference pricing mechanisms and obtain an upper bound to the maximum energy efficiency problem including the inter-cell interference contributions. The performance of the proposed algorithm is studied in a Long Term Evolution (LTE) system. Our simulation results demonstrate that the proposed algorithm provides substantial improvements in the energy efficiency and throughput of the network. It is also shown that interference pricing provides only marginal improvements over the proposed algorithm.

Cosmic web reconstruction through density ridges: method and algorithm

The detection and characterization of filamentary structures in the cosmic web allows cosmologists to constrain parameters that dictates the evolution of the Universe. While many filament estimators have been proposed, they generally lack estimates of uncertainty, reducing their inferential power. In this paper, we demonstrate how one may apply the Subspace Constrained Mean Shift (SCMS) algorithm (Ozertem and Erdogmus (2011); Genovese et al. (2012)) to uncover filamentary structure in galaxy data. The SCMS algorithm is a gradient ascent method that models filaments as density ridges, one-dimensional smooth curves that trace high-density regions within the point cloud. We also demonstrate how augmenting the SCMS algorithm with bootstrap-based methods of uncertainty estimation allows one to place uncertainty bands around putative filaments. We apply the SCMS method to datasets sampled from the P3M N-body simulation, with galaxy number densities consistent with SDSS and WFIRST-AFTA and to LOWZ and CMASS data from the Baryon Oscillation Spectroscopic Survey (BOSS). To further assess the efficacy of SCMS, we compare the relative locations of BOSS filaments with galaxy clusters in the redMaPPer catalog, and find that redMaPPer clusters are significantly closer (with p-values $< 10^{-9}$) to SCMS-detected filaments than to randomly selected galaxies.

Deep multilayer multiple kernel learning

Multiple kernel learning (MKL) approach has been proposed for kernel methods and has shown high performance for solving some real-world applications. It consists on learning the optimal kernel from one layer of multiple predefined kernels. Unfortunately, this approach is not rich enough to solve relatively complex problems. With the emergence and the success of the deep learning concept, multilayer of multiple kernel learning (MLMKL) methods were inspired by the idea of deep architecture. They are introduced in order to improve the conventional MKL methods. Such architectures tend to learn deep kernel machines by exploring the combinations of multiple kernels in a multilayer structure. However, existing MLMKL methods often have trouble with the optimization of the network for two or more layers. Additionally, they do not always outperform the simplest method of combining multiple kernels (i.e., MKL). In order to improve the effectiveness of MKL approaches, we introduce, in this paper, a novel backpropagation MLMKL framework. Specifically, we propose to optimize the network over an adaptive backpropagation algorithm. We use the gradient ascent method instead of dual objective function, or the estimation of the leave-one-out error. We test our proposed method through a large set of experiments on a variety of benchmark data sets. We have successfully optimized the system over many layers. Empirical results over an extensive set of experiments show that our algorithm achieves high performance compared to the traditional MKL approach and existing MLMKL methods.

A Study on Multi-Scale Kernel Optimisation via Centered Kernel-Target Alignment

Kernel mapping is one of the most widespread approaches to intrinsically deriving nonlinear classifiers. With the aim of better suiting a given dataset, different kernels have been proposed and different bounds and methodologies have been studied to optimise them. We focus on the optimisation of a multi-scale kernel, where a different width is chosen for each feature. This idea has been barely studied in the literature, although it has been shown to achieve better performance in the presence of heterogeneous attributes. The large number of parameters in multi-scale kernels makes it computationally unaffordable to optimise them by applying traditional cross-validation. Instead, an analytical measure known as centered kernel-target alignment (CKTA) can be used to align the kernel to the so-called ideal kernel matrix. This paper analyses and compares this and other alternatives, providing a review of the literature in kernel optimisation and some insights into the usefulness of multi-scale kernel optimisation via CKTA. When applied to the binary support vector machine paradigm (SVM), the results using 24 datasets show that CKTA with a multi-scale kernel leads to the construction of a well-defined feature space and simpler SVM models, provides an implicit filtering of non-informative features and achieves robust and comparable performance to other methods even when using random initialisations. Finally, we derive some considerations about when a multi-scale approach could be, in general, useful and propose a distance-based initialisation technique for the gradient-ascent method, which shows promising results.

A DEC-MDP model for joint uplink/downlink resource management in OFDMA-based networks

With the advent of mobile services with asymmetric and symmetric quality of service (QoS) requirements, traditional single link resource allocation techniques have started to show some limitations in handling the complex requirements. To address these issues, joint uplink/downlink resource management approaches were recently introduced where both communications links are jointly considered in the resource management process. One direct consequence of this coupling is a modification of the underlying queueing behavior since the decision making process in one direction in terms of transmission rate now depends on the performance achieved in the opposite direction. In this paper, we present a modeling approach of the decision making process that takes place under the joint uplink/downlink resource management framework. Using decentralized Markov decision processes (DEC-MDP) as a model and gradient ascent methods as an optimization technique, we formulate and solve the joint uplink/downlink decision making process. The uplink and downlink of each user are considered as agents. Assuming certain subcarrier and power allocation schemes, we investigate the resource usage in the uplink and downlink to achieve a certain delay balancing constraint where the total delay in the uplink and downlink is bound by a pre-determined threshold. The approach followed starts by modeling the problem in hand using DEC-MDPs. After discussing the different aspects of the model, the solution using gradient ascent is described. Simulation results illustrate the different dimensions of the problem and their impact on the resource management process.

Adaptive phase denoising based on first order local polynomial analysis

This paper concentrates on the optical interferometry wrapped phase denoising method. A new adaptive denoising algorithm based on the local polynomial approximation is researched. Firstly, the local polynomial analysis is studied and the polynomial parameters are estimated by the proposed method which combines the Windowed Fast Fourier Transform with the gradient ascent method. Then the adaptive data window sizes are determined by the proposed largest window size with acceptable hypothesis method. One improvement is that our algorithm uses the first order polynomial method to select adaptive window sizes and to do the phase approximation. At last, a set of experiments are shown to prove that the proposed method can compromise between preventing noise and preserving original phase details.

Beyond normality: A cross moment-stochastic user equilibrium model

The Stochastic User Equilibrium (SUE) model predicts traffic equilibrium flow assuming that users choose their perceived maximum utility paths (or perceived shortest paths) while accounting for the effects of congestion that arise due to users sharing links. Inspired by recent work on distributionally robust optimization, specifically a Cross Moment (CMM) choice model, we develop a new SUE model that uses the mean and covariance information on path utilities but does not assume the particular form of the distribution. Robustness to distributional assumptions is obtained in this model by minimizing the worst-case expected cost over all distributions with fixed two moments. We show that under mild conditions, the CMM-SUE (Cross Moment-Stochastic User Equilibrium) exists and is unique. By combining a simple projected gradient ascent method to evaluate path choice probabilities with a gradient descent method to find flows, we show that the CMM-SUE is efficiently computable. CMM-SUE provides both modeling flexibility and computational advantages over approaches such as the well-known MNP-SUE (Multinomial Probit-Stochastic User Equilibrium) model that require distributional (normality) assumptions to model correlation effects from overlapping paths. In particular, it avoids the use of simulation methods employed in computations for the distribution-based MNP-SUE model. Preliminary computational results indicate that CMM-SUE provides a practical distributionally robust alternative to MNP-SUE.

Optimized Phase-Sensitive Inversion Recovery for MRI Contrast Manipulation

In this paper, we propose a practical optimized phase-sensitive inversion recovery approach for longitudinal contrast enhancement of tissues. With the usage of double dynamic range of longitudinal magnetization, the contrast of tissues can be maximized with an optimal inversion pulse, which is optimized by an extended version of optimal-control-based gradient ascent method. Moreover, to demonstrate this optimized contrast, a modified image reconstruction of region-growing spin reversal operation is presented. Simulation experiments show that even for tissues with same \(T_{1}\) value, the contrast can be increased up to 28 \(\%\) compared with the saturation contrast enhancement. It has a great potential for enhancing contrast of tissues, and in particular, of limited intrinsic different tissues.

A multi-sensor RSS spatial sensing-based robust stochastic optimization algorithm for enhanced wireless tethering.

The reliability of wireless communication in a network of mobile wireless robot nodes depends on the received radio signal strength (RSS). When the robot nodes are deployed in hostile environments with ionizing radiations (such as in some scientific facilities), there is a possibility that some electronic components may fail randomly (due to radiation effects), which causes problems in wireless connectivity. The objective of this paper is to maximize robot mission capabilities by maximizing the wireless network capacity and to reduce the risk of communication failure. Thus, in this paper, we consider a multi-node wireless tethering structure called the “server-relay-client” framework that uses (multiple) relay nodes in between a server and a client node. We propose a robust stochastic optimization (RSO) algorithm using a multi-sensor-based RSS sampling method at the relay nodes to efficiently improve and balance the RSS between the source and client nodes to improve the network capacity and to provide redundant networking abilities. We use pre-processing techniques, such as exponential moving averaging and spatial averaging filters on the RSS data for smoothing. We apply a receiver spatial diversity concept and employ a position controller on the relay node using a stochastic gradient ascent method for self-positioning the relay node to achieve the RSS balancing task. The effectiveness of the proposed solution is validated by extensive simulations and field experiments in CERN facilities. For the field trials, we used a youBot mobile robot platform as the relay node, and two stand-alone Raspberry Pi computers as the client and server nodes. The algorithm has been proven to be robust to noise in the radio signals and to work effectively even under non-line-of-sight conditions.