Regularized Least-squares Algorithm Research Articles

Hydraulic fracturing distributed acoustic sensing monitoring data source mechanism inversion: A Hessian-based method

Distributed acoustic sensing (DAS) microseismic monitoring during hydraulic fracturing provides microseismic data with high spatial samplings for fracturing zones. However, sources located perpendicular to the single horizontal well of the DAS acquisition system have limited source-receiver geometries with extremely poor azimuthal coverage, resulting in high uncertainty in source mechanism inversion. To address this problem, we introduce the Hessian matrix, which governs the blurring effect caused by the source-receiver geometry, into the DAS microseismic source mechanism inversion. Our Hessian-based source mechanism inversion method consists of three main steps: (1) construct the Hessian matrix of the source mechanism based on the source-receiver geometry, (2) obtain an initial source mechanism using a conventional source mechanism inversion method, and (3) update the initial source mechanism using a Hessian-based [Formula: see text]-regularized least-squares algorithm. We assess the robustness of our method using synthetic DAS microseismic data with the consideration of noise, source location error, and different regularization parameters, and we compare the results with those of the conventional method. The results demonstrate that the Hessian-based method has a remarkable ability to mitigate the blurring effect of the Hessian matrix caused by limited DAS source-receiver geometry with poor azimuthal coverage, thereby reducing the uncertainty of the inverted source mechanism even in the presence of real noise and/or source location error. Finally, we use our method to invert the source mechanism of a real DAS microseismic event acquired during hydraulic fracturing. Our Hessian-based method provides low uncertainty in the source mechanism inversion of the real DAS microseismic data.

Multi-Zone Active Noise Control Strategy for the Scattered Sound Control of an Infinite Rigid Cylinder

Active noise control can be used to reduce the scattered sound of a reflecting object to make it invisible to incident acoustic waves. For the multi-zone active noise control of scattered sound from an infinite rigid cylinder, an active control strategy is proposed that combines the least absolute shrinkage and selection operator (LASSO) algorithm with constraint points and regularized least squares (RLS) algorithm. The proposed control strategy is used to promote control performance through optimizing the secondary loudspeaker placement of an active noise control system. Compared with the RLS algorithm employing the uniformly placed loudspeakers and the traditional LASSO algorithm, the proposed strategy has better reduction performance both in the forward-scattered and backward-scattered sound target areas, and there is less sound amplification in the far field. From 400 Hz–1100 Hz, the proposed strategy provides a 5 dB–16 dB reduction performance advantage in the target area compared to the RLS algorithm employing uniformly placed loudspeakers.

Learning With Selected Features.

The coming big data era brings data of unprecedented size and launches an innovation of learning algorithms in statistical and machine-learning communities. The classical kernel-based regularized least-squares (RLS) algorithm is excluded in the innovation, due to its computational and storage bottlenecks. This article presents a scalable algorithm based on subsampling, called learning with selected features (LSF), to reduce the computational burden of RLS. Almost the optimal learning rate together with a sufficient condition on selecting kernels and centers to guarantee the optimality is derived. Our theoretical assertions are verified by numerical experiments, including toy simulations, UCI standard data experiments, and a real-world massive data application. The studies in this article show that LSF can reduce the computational burden of RLS without sacrificing its generalization ability very much.

Recovery of a Compressed Sensing CT Image Using a Smooth Re-weighted Function- Regularized Least-Squares Algorithm

It is challenging to recover the required compressed CT (Computed Tomography, CT) image, which is got by transferred through the internet or is stored in a signal library after being compressed. We present a recovery method for compressed sensing CT images. At present, minimizing 0-norm, 1-norm and p-norm is used to recover compressed sensing signals. However, sometimes 0-norm is an NP problem, 1-norm has no solution in theory and p-norm is not a convex function. We introduce a recovery method of compressed sensing signal based on regularized smooth convex optimization. In order to avoid solving the non-convex optimization problems and no solution condition, a convex function is designed as the objective function of optimization to fit 0-norm of signal and a fast iterative shrinkage-thresholding algorithm is proposed to find solution with the convergence speed is quadratic convergence. Experimental results show that our method has a sound recovery effect and is well suitable for processing big data of compressed CT images.

Consistency-based rectification of nonrigid registrations

We present a technique to rectify nonrigid registrations by improving their group-wise consistency, which is a widely used unsupervised measure to assess pair-wise registration quality. While pair-wise registration methods cannot guarantee any group-wise consistency, group-wise approaches typically enforce perfect consistency by registering all images to a common reference. However, errors in individual registrations to the reference then propagate, distorting the mean and accumulating in the pair-wise registrations inferred via the reference. Furthermore, the assumption that perfect correspondences exist is not always true, e.g., for interpatient registration. The proposed consistency-based registration rectification (CBRR) method addresses these issues by minimizing the group-wise inconsistency of all pair-wise registrations using a regularized least-squares algorithm. The regularization controls the adherence to the original registration, which is additionally weighted by the local postregistration similarity. This allows CBRR to adaptively improve consistency while locally preserving accurate pair-wise registrations. We show that the resulting registrations are not only more consistent, but also have lower average transformation error when compared to known transformations in simulated data. On clinical data, we show improvements of up to 50% target registration error in breathing motion estimation from four-dimensional MRI and improvements in atlas-based segmentation quality of up to 65% in terms of mean surface distance in three-dimensional (3-D) CT. Such improvement was observed consistently using different registration algorithms, dimensionality (two-dimensional/3-D), and modalities (MRI/CT).

Application of an Empirical FBRM Model to Estimate Crystal Size Distributions in Batch Crystallization

Monitoring crystal size distributions in situ is a challenge in crystallization engineering. Such a capability provides comprehensive information on the crystallization, which can be used to improve the quality of products. We developed an empirical focused-beam reflectance measurement (FBRM) model to provide such estimates. By experimentally fixing the CSD as a system input and the chord-length distribution (CLD) from the FBRM as the output, we constructed the model as a linear transformation from CSD to CLD. A regularized least-squares algorithm, which considers the mass balances on solid and liquid phases, was used to estimate the CSD. The results of batch cooling crystallization experiments show that estimates of the evolution of CSD are in accordance with the general understanding of crystallization kinetics, and the CSD estimates of final product agreed with the CSD obtained from sieving. The simplicity and practicality of the model make it a significant enhancement to the use of FBRM in monitoring, simulating, and controlling crystallization processes.

Multi-label learning under feature extraction budgets

We consider the problem of learning sparse linear models for multi-label prediction tasks under a hard constraint on the number of features. Such budget constraints are important in domains where the acquisition of the feature values is costly. We propose a greedy multi-label regularized least-squares algorithm that solves this problem by combining greedy forward selection search with a cross-validation based selection criterion in order to choose, which features to include in the model. We present a highly efficient algorithm for implementing this procedure with linear time and space complexities. This is achieved through the use of matrix update formulas for speeding up feature addition and cross-validation computations. Experimentally, we demonstrate that the approach allows finding sparse accurate predictors on a wide range of benchmark problems, typically outperforming the multi-task lasso baseline method when the budget is small.

Study on SOC Estimation Based on Circular Optimization for RBF Neural Network

This paper proposed a circular particle swarm optimization least squares (CPSOLS) method which is consisted of the regularized least squares (RLS) method and the adaptive particle swarm optimization (APSO) algorithm. The RLS algorithm optimized the parameters of the RBF network, aiming at the phenomenon of RLS trapping in the local minimum, introduced the penalty factor and used the global optimization ability of the particle swarm optimization algorithm to make it out of the local minimum; simplified the structure of the RBF network and improved the generalization ability of the network. The APSO algorithm weakened the precocious converge phenomena of the particle swarm optimization algorithm, adopted the adaptive selection of the nonlinear dynamic inertia weight which is guided by the control factor of the battery external characteristic temperature parameters, optimized the link weight of the RBF network, improved the state of charge (SOC) estimation accuracy and real-time performance of the RBF network. Using the Arbin multifunctional battery test system BT2000 to collect the sample data of the battery external characteristic parameters, and using the sample data to train and optimize the RBF neural network, and estimate the SOC of the batteries. The results showed that the optimized RBF network improved the SOC estimation accuracy and real-time performance.

Functional Extended Redundancy Analysis.

We propose a functional version of extended redundancy analysis that examines directional relationships among several sets of multivariate variables. As in extended redundancy analysis, the proposed method posits that a weighed composite of each set of exogenous variables influences a set of endogenous variables. It further considers endogenous and/or exogenous variables functional, varying over time, space, or other continua. Computationally, the method reduces to minimizing a penalized least-squares criterion through the adoption of a basis function expansion approach to approximating functions. We develop an alternating regularized least-squares algorithm to minimize this criterion. We apply the proposed method to real datasets to illustrate the empirical feasibility of the proposed method.

History Matching With Learned Sparse Dictionaries

This article, written by Senior Technology Editor Dennis Denney, contains highlights of paper SPE 133654, ’History Matching With Learned Sparse Dictionaries,’ by Mohammad-Reza M. Khaninezhad, Behnam Jafarpour, SPE, and Lianlin Li, Texas A&M University, prepared for the 2010 SPE Annual Technical Conference and Exhibition, Florence, Italy, 19-22 September. The paper has not been peer reviewed. History matching of heterogeneous hydrocarbon reservoirs is carried out to construct predictive reservoir models. Estimating individual-gridblock heterogeneous-reservoir properties from limited static data and spatially averaged dynamic-production measurements inherently leads to an ill-posed inverse problem that can have several nonunique solutions. To achieve solution sparsity, an iteratively reweighted-sparsity regularized least-squares algorithm is used to combine and weigh a small subset of relevant model elements selectively from a learned dictionary that explains the observed data. Introduction History matching of hydrocarbon-reservoir models is required to integrate production data that constrain uncertain previous descriptions of model parameters (i.e., spatially distributed petrophysical properties). The main goal of history matching is to construct updated models capable of predicting future performance of the reservoir more accurately. Reconstructing petrophysical properties is a challenging inverse problem because of data scarcity and because these properties are heterogeneous. When spatial discretization of the forward-simulation model is used to describe unknown spatial parameters in history matching, the problem typically becomes ill posed because the number of unknown parameters overwhelmingly exceeds available observations. Consequently, several nonunique solutions can exist that respect available observations but exhibit different spatial continuity and flow predictions. Several techniques can regularize ill-posed history-matching problems. Usually, regularization involves incorporating additional prior assumptions to eliminate implausible solutions and improve the stability of solution algorithms. Reparameterization Reparameterization of the original spatial problem is a regularization approach for ill-posed inverse problems in which a reduced number of parameters is used to approximate the unknown-petrophysical-property maps. Reparameterization is facilitated when reliable a priori information about structural attributes of the solution is available. Mathematical transformations provide an effective reparameterization approach when the properties of the unknown parameters are exploited better in the transform domain. For example, principal-component analysis (PCA) is effective in reparameterizing correlated parameters by elimination of their correlation (redundancy). Other popular transform-domain reparameterization methods that have compression properties include the discrete-cosine transform (DCT) and the discrete-wavelet transform (DWT).

Variable regularized least-squares algorithm: One-step-ahead cost function with equivalent optimality

This paper proposes a new variable regularized least-squares (VR-LS) algorithm by recursively constructing a weighting scalar of the regularized least-squares (LS) cost function. Since the recursive LS (RLS) algorithm provides the best performances by all of VR-LS algorithms, the design objective of the weighting scalar is chosen such that equivalent optimality is ensured between one-step-ahead cost functions of the RLS and of the VR-LS algorithm. The proposed VR-LS algorithm functions similarly as the RLS with uncorrelated inputs; however, this is not the case with colored (correlated) inputs. Therefore, a conventional filtering technique is applied to both on the inputs and on the desired signals so as to obtain whitened inputs. This enables the proposed algorithm handle the case of correlated inputs.

Learning intransitive reciprocal relations with kernel methods

In different fields like decision making, psychology, game theory and biology, it has been observed that paired-comparison data like preference relations defined by humans and animals can be intransitive. Intransitive relations cannot be modeled with existing machine learning methods like ranking models, because these models exhibit strong transitivity properties. More specifically, in a stochastic context, where often the reciprocity property characterizes probabilistic relations such as choice probabilities, it has been formally shown that ranking models always satisfy the well-known strong stochastic transitivity property. Given this limitation of ranking models, we present a new kernel function that together with the regularized least-squares algorithm is capable of inferring intransitive reciprocal relations in problems where transitivity violations cannot be considered as noise. In this approach it is the kernel function that defines the transition from learning transitive to learning intransitive relations, and the Kronecker-product is introduced for representing the latter type of relations. In addition, we empirically demonstrate on two benchmark problems, one in game theory and one in theoretical biology, that our algorithm outperforms methods not capable of learning intransitive reciprocal relations.

Towards automated classification of intensive care nursing narratives

Background Nursing narratives are an important part of patient documentation, but the possibilities to utilize them in the direct care process are limited due to the lack of proper tools. One solution to facilitate the utilization of narrative data could be to classify them according to their content. Objectives Our objective is to address two issues related to designing an automated classifier: domain experts’ agreement on the content of classes Breathing, Blood Circulation and Pain, as well as the ability of a machine-learning-based classifier to learn the classification patterns of the nurses. Methods The data we used were a set of Finnish intensive care nursing narratives, and we used the regularized least-squares (RLS) algorithm for the automatic classification. The agreement of the nurses was assessed by using Cohen's κ, and the performance of the algorithm was measured using area under ROC curve (AUC). Results On average, the values of κ were around 0.8. The agreement was highest in the class Blood Circulation, and lowest in the class Breathing. The RLS algorithm was able to learn the classification patterns of the three nurses on an acceptable level; the values of AUC were generally around 0.85. Conclusions Our results indicate that the free text in nursing documentation can be automatically classified and this can offer a way to develop electronic patient records.

On regularization algorithms in learning theory

In this paper we discuss a relation between Learning Theory and Regularization of linear ill-posed inverse problems. It is well known that Tikhonov regularization can be profitably used in the context of supervised learning, where it usually goes under the name of regularized least-squares algorithm. Moreover, the gradient descent algorithm was studied recently, which is an analog of Landweber regularization scheme. In this paper we show that a notion of regularization defined according to what is usually done for ill-posed inverse problems allows to derive learning algorithms which are consistent and provide a fast convergence rate. It turns out that for priors expressed in term of variable Hilbert scales in reproducing kernel Hilbert spaces our results for Tikhonov regularization match those in Smale and Zhou [Learning theory estimates via integral operators and their approximations, submitted for publication, retrievable at 〈 http://www.tti-c.org/smale.html 〉 , 2005] and improve the results for Landweber iterations obtained in Yao et al. [On early stopping in gradient descent learning, Constructive Approximation (2005), submitted for publication]. The remarkable fact is that our analysis shows that the same properties are shared by a large class of learning algorithms which are essentially all the linear regularization schemes. The concept of operator monotone functions turns out to be an important tool for the analysis.

Optimal Rates for the Regularized Least-Squares Algorithm

We develop a theoretical analysis of the performance of the regularized least-square algorithm on a reproducing kernel Hilbert space in the supervised learning setting. The presented results hold in the general framework of vector-valued functions; therefore they can be applied to multitask problems. In particular, we observe that the concept of effective dimension plays a central role in the definition of a criterion for the choice of the regularization parameter as a function of the number of samples. Moreover, a complete minimax analysis of the problem is described, showing that the convergence rates obtained by regularized least-squares estimators are indeed optimal over a suitable class of priors defined by the considered kernel. Finally, we give an improved lower rate result describing worst asymptotic behavior on individual probability measures rather than over classes of priors.

Deconvolution of 2D coincident Doppler broadening spectroscopy using the Richardson–Lucy algorithm

Coincident Doppler Broadening Spectroscopy (CDBS) measurements are popular in positron solid-state studies of materials. By utilizing the instrumental resolution function obtained from a gamma line close in energy to the 511 keV annihilation line, it is possible to significantly enhance the quality of the CDBS spectra using deconvolution algorithms. In this paper, we compare two algorithms, namely the Non-Negativity Least Squares (NNLS) regularized method and the Richardson–Lucy (RL) algorithm. The latter, which is based on the method of maximum likelihood, is found to give superior results to the regularized least-squares algorithm and with significantly less computer processing time.

Learning from Examples as an Inverse Problem

Many works related learning from examples to regularization techniques for inverse problems, emphasizing the strong algorithmic and conceptual analogy of certain learning algorithms with regularization algorithms. In particular it is well known that regularization schemes such as Tikhonov regularization can be effectively used in the context of learning and are closely related to algorithms such as support vector machines. Nevertheless the connection with inverse problem was considered only for the discrete (finite sample) problem and the probabilistic aspects of learning from examples were not taken into account. In this paper we provide a natural extension of such analysis to the continuous (population) case and study the interplay between the discrete and continuous problems. From a theoretical point of view, this allows to draw a clear connection between the consistency approach in learning theory and the stability convergence property in ill-posed inverse problems. The main mathematical result of the paper is a new probabilistic bound for the regularized least-squares algorithm. By means of standard results on the approximation term, the consistency of the algorithm easily follows.

Noise suppression of point spread functions and its influence on deconvolution of three‐dimensional fluorescence microscopy image sets

The point spread function (PSF) is of central importance in the image restoration of three-dimensional image sets acquired by an epifluorescent microscope. Even though it is well known that an experimental PSF is typically more accurate than a theoretical one, the noise content of the experimental PSF is often an obstacle to its use in deconvolution algorithms. In this paper we apply a recently introduced noise suppression method to achieve an effective noise reduction in experimental PSFs. We show with both simulated and experimental three-dimensional image sets that a PSF that is smoothed with this method leads to a significant improvement in the performance of deconvolution algorithms, such as the regularized least-squares algorithm and the accelerated Richardson-Lucy algorithm.

Model Selection for Regularized Least-Squares Algorithm in Learning Theory

We investigate the problem of model selection for learning algorithms depending on a continuous parameter. We propose a model selection procedure based on a worst-case analysis and on a data-independent choice of the parameter. For the regularized least-squares algorithm we bound the generalization error of the solution by a quantity depending on a few known constants and we show that the corresponding model selection procedure reduces to solving a bias-variance problem. Under suitable smoothness conditions on the regression function, we estimate the optimal parameter as a function of the number of data and we prove that this choice ensures consistency of the algorithm.

THE GENERALIZED LEAST-SQUARES SYSTEM IDENTIFICATION SUBJECT TO UNKNOWN BOUNDED NOISE

On the base of the least-squares technique the universal method of regularized identification is developed. For any given input and output in the presence of the worst-case bounded noise it provides the convergent parameter estimate when the order of stable model is increased. This implies that even in the case when available information is not sufficient to identify the system it is possible to evaluate an approximate model using the approach offered in the paper. The recurrent form of regularized least-squares algorithm with adaptive regularization depending on incoming data is constructed.