Regularized Square Research Articles

Emotional Response using Power Spectrum Approach

The objective of this study is to detect affective response of children to facial expression based on alpha power density of brain activity. Electroencephalography data were collected from 10 typical children. The alpha power temporal information of active brain regions was extracted. Performance of the power spectrum feature was evaluated in emotion recognition process using K nearest neighbor, a regularized least square and multilayer perceptron classifier. A statistical analysis indicated right alpha activity during negative and calm emotional states. Statistical results showed significant difference between rest conditions and emotional state. The best accuracy we got to detect emotional states is by using regularized least square that is 70%.

Proposals for local basis selection for the sparse representation-based classifier

In the domains of pattern recognition and computer vision, sparse representation classifier and its variants are considered as powerful classifiers. However, due to the use of sparse coding in most of its variants, classifying test samples is computationally expensive. Thus, it is not practical for scenarios demanding fast classification. For this reason, a two-phase coding classifier based on classic regularized least square was proposed recently. A significant limitation of this classifier is the fact that the number of local bases that should be handed over to the next coding phase should be specified manually. This paper overcomes this main limitation and proposes three data-driven schemes allowing an automatic estimation of the optimal size of the local bases. Experiments conducted on five image datasets show that the introduced schemes, despite their simplicity, can improve the performance of the two-phase linear coding classifier adopting ad hoc choices for the number of local bases.

Comparison of ANN and RKS approaches to model SCC strength

Self compacting concrete (SCC) is a high performance concrete that has high flowability and can be used in heavily reinforced concrete members with minimal compaction segregation and bleeding. The mix proportioning of SCC is highly complex and large number of trials are required to get the mix with the desired properties resulting in the wastage of materials and time. The research on SCC has been highly empirical and no theoretical relationships have been developed between the mixture proportioning and engineering properties of SCC. In this work effectiveness of artificial neural network (ANN) and random kitchen sink algorithm(RKS) with regularized least square algorithm(RLS) in predicting the split tensile strength of the SCC is analysed. Random kitchen sink algorithm is used for mapping data to higher dimension and classification of this data is done using Regularized least square algorithm. The training and testing data for the algorithm was obtained experimentally using standard test procedures and materials available. Total of 40 trials were done which were used as the training and testing data. Trials were performed by varying the amount of fine aggregate, coarse aggregate, dosage and type of super plasticizer and water. Prediction accuracy of the ANN and RKS model is checked by comparing the RMSE value of both ANN and RKS. Analysis shows that eventhough the RKS model is good for large data set, its prediction accuracy is as good as conventional prediction method like ANN so the split tensile strength model developed by RKS can be used in industries for the proportioning of SCC with tailor made property.

Modelling the minislump spread of superplasticized PPC paste using RLS with the application of Random Kitchen sink

Super plasticizers(SPs) are added to the concrete to improve its workability with out changing the water cement ratio. Property of fresh concrete is mainly governed by the cement paste which depends on the dispersion of cement particle. Cement dispersive properties of the SP depends up on its dosage and the family. Mini slump spread diameter with different dosages and families of SP is taken as the measure of workability characteristic of cement paste chosen for measuring the rheological properties of cement paste. The main purpose of this study includes measure the dispersive ability of different families of SP by conducting minislump test and model the minislump spread diameter of the super plasticized Portland Pozzolona Cement (PPC)paste using regularized least square (RLS) approach along with the application of Random kitchen sink (RKS) algorithm. For preparing test and training data for the model 287 different mixes were prepared in the laboratory at a water cement ratio of 0.37 using four locally available brand of Portland Pozzolona cement (PPC) and SP belonging to four different families. Water content, cement weight and amount of SP (by considering it as seven separate input based on their family and brand) were the input parameters and mini slump spread diameter was the output parameter for the model. The variation of predicted and measured values of spread diameters were compared and validated. From this study it was observed that, the model could effectively predict the minislump spread of cement paste

Enhanced regularized least square based discriminative projections for feature extraction

The regularized least square based discriminative projections (RLSDP) for extracting features was recently proposed, which aims to seek discriminant projection directions that maximize the between-class scatter and minimize the within-class compactness. However, in RLSDP, the retrieval samples are reconstructed by the coefficients only associated with the same class, and may have large errors. Moreover, the distances between each sample and other within-class samples characterize the most important within-class compactness information, and are not minimized in RLSDP. To deal with the above two problems, we propose an enhanced regularized least square based discriminative projections (ERLSDP). ERLSDP utilizes all the related coefficients of each sample for reconstruction and explicitly minimizes the distances between all the within-class samples, and thus it has better reconstruction accuracy and more discriminating power than that of RLSDP. Experimental results demonstrate that ERLSDP gets a clear improvement over RLSDP when the training sample size is small.

A nonconvex nonsmooth regularization method for compressed sensing and low rank matrix completion

In this paper, a nonconvex and nonsmooth method for compressed sensing and low-rank matrix completion is studied. The proposed model is formulated as nonconvex regularized least square optimization problem. At first, an alternating minimization scheme is developed in which the problem can be decomposed into three subproblems, two of them are convex and the remaining one is smooth. Then, the convergence of the sequence which is generated by the alternating minimization algorithm is proved. In addition, some recovery guarantees are also analyzed. Finally, various numerical simulations are performed to test the efficiency of the method.

Discriminative local collaborative representation for online object tracking

Sparse representation has been widely applied to object tracking. However, most sparse representation based trackers only use the holistic template to encode the candidates, where the discriminative information to separate the target from the background is ignored. In addition, the sparsity assumption with the l1 norm minimization is computationally expensive. In this paper, we propose a robust discriminative local collaborative (DLC) representation algorithm for online object tracking. DLC collaboratively uses the local image patches of both the target templates and the background ones to encode the candidates by an efficient local regularized least square solver with the l2 norm minimization, where the feature vectors are obtained by employing an effective discriminative-pooling method. Furthermore, we formulate the tracking as a discriminative classification problem, where the classifier is online updated by using the candidates predicted according to the residuals of their local patches. To adapt to the appearance changes, we iteratively update the dictionary with the foreground and background templates from the current frame and take occlusions into account as well. Experimental results demonstrate that our proposed algorithm performs favorably against the state-of-the-art trackers on several challenging video sequences.

A regularized least square based discriminative projections for feature extraction

In this paper, we present a regularized least square based discriminative projections (RLSDP) method for feature extraction. First, we show that both sparse representation based classifier (SRC) and collaborative representation based classification (CRC) are regularized least square in nature. Second, a regularized least square based graph embedding framework (RLSGE) is constructed. Third, a RLSGE based feature extraction method is given, named regularized least square based discriminant projections (RLSDP). In RLSDP, the within-class compactness information is characterized by the reconstruction residual from the same class, which is consistent with the idea of reconstruction; the between-class separability information is characterized by the between-class scatter matrix like Fisher LDA. RLSDP is much faster than SPP since RLSDP adopts the L2 norm constraint while SPP adopts the L1 norm constraint. The experimental results on AR face database, FERET face database, and the PolyU FKP database demonstrate that RLSDP works well in feature extraction and has a great recognition performance.

An Alternating Direction Method with Continuation for Nonconvex Low Rank Minimization

In this paper we consider a nonconvex model of recovering low-rank matrices from the noisy measurement. The problem is formulated as a nonconvex regularized least square optimization problem, in which the rank function is replaced by a matrix minimax concave penalty function. An alternating direction method with a continuation (ADMc) technique (on the regularization parameter) is proposed to solve this nonconvex low rank matrix recovery problem. Moreover, under some mild assumptions, the convergence behavior of the alternating direction method for the proposed nonconvex problems is proved. Finally, comprehensive numerical experiments show that the proposed nonconvex model and the ADM algorithm are competitive with the state-of-the-art models and algorithms in terms of efficiency and accuracy.

New Improved Algorithms for Compressive Sensing Based on $\ell_{p}$ Norm

A new algorithm for the reconstruction of sparse signals, which is referred to as the lp-regularized least squares ( lp-RLS) algorithm, is proposed. The new algorithm is based on the minimization of a smoothed lp-norm regularized square error with p <; 1 . It uses a conjugate-gradient (CG) optimization method in a sequential minimization strategy that involves a two-parameter continuation technique. An improved version of the new algorithm is also proposed, which entails a bisection technique that optimizes an inherent regularization parameter. Extensive simulation results show that the new algorithm offers improved signal reconstruction performance and requires reduced computational effort relative to several state-of-the-art competing algorithms. The improved version of the lp-RLS algorithm offers better performance than the basic version, although this is achieved at the cost of increased computational effort.

Regularized least square discriminant projection and feature selection

Conventional graph embedding framework uses the Euclidean distance to determine the similarities of neighbor samples, which causes the graph structure to be sensitive to outliers and lack physical interpretation. Moreover, the graph construction suffers from the difficulty of neighbor parameter selection. Although sparse representation (SR) based graph embedding methods can automatically select the neighbor parameter, the computational cost of SR is expensive. On the other hand, most discriminant projection methods fail to perform feature selection. In this paper, we present a novel joint discriminant analysis and feature selection method that employs regularized least square for graph construction and l 2,1 -norm minimization on projection matrix for feature selection. Specifically, our method first uses the regularized least square coefficients to measure the intraclass and interclass similarities from the viewpoint of reconstruction. Based on this graph structure, we formulate an object function with scatter difference criterion for learning the discriminant projections, which can avoid the small sample size problem. Simultaneously, the l 2,1 -norm minimization on projection matrix is applied to gain row-sparsity for selecting useful features. Experiments on two face databases (ORL and AR) and COIL-20 object database demonstrate that our method not only achieves better classification performance, but also has lower computational cost than SR.

Observer design for constrained nonlinear systems with application to secure communication

In this paper, a constrained regularized least square (RLS) state estimator is developed for deterministic discrete-time nonlinear dynamical systems subject to a set of equality and/or inequality constraints. The stability of the estimation error is rigorously analyzed. The proposed estimator is then used to handle the important problem of secure communication. At the transmitting end, the output of the constrained unified chaotic system is used as a chaotic mask to achieve a satisfactory and typical secure communication scheme. The encrypted data signal is injected into the transmitter and simultaneously transmitted to the receiver through a public channel. At the receiving end, the constrained RLS estimator is used to reconstruct the states of the constrained unified chaotic system. Simulation results are presented to show the impact of the imposed constraints on the waveform and the pattern of the generated chaotic signal as well as the ability of the proposed estimator to synchronize the actual and estimated states of the constrained unified chaotic system. Moreover, the proposed estimator is applied to recover discrete signals such as digital images where computer simulation results are provided to show the effectiveness of the proposed estimation scheme.

The Regularization Method with Morbid Problems of Equality Constraints

The difference between the parameters and reasonable equality constraints can significantly improve the precision of classical least squares solution to eliminate the rank loss free network problems, and also can improve the pathological solution of the model, which presents the measurement adjustment equality constraints attached pathological model. This paper establishes the equality constraints morbid Model Adjustment regularization criteria, to give the solution of the model, and gives the statistical analysis to understand the nature. Based on this data, and giving full consideration of inevitable gross errors, it puts forward robust regularized factor mean square error minimization criteria and numerical solution. A variety of programs the calculation results show that the method not only can eliminate the adverse effects of the matrix morbid, but also can be able to overcome the presence of gross errors caused by instability, which is a good estimate.

ODVBA-C: Optimally-Discriminative Voxel-Based Analysis of Continuous Variables.

In this paper, we propose a new method that utilizes a novel spatially adaptive scheme for detection of multivariate neuroimaging patterns relating to a continuous subject-level variable, aiming to effectively determine the optimal spatially adaptive filtering of neuroimaging data from the persepective of finding relationships between imaging and continues (e.g. clinical and cognitive) variables. Analyses employ local pattern analysis using regularized least square regression with nonnegativity constraints within a spatial neighborhood around each voxel. Within each neighborhood, we determine the optimal regression coefficients that relate local patterns to the continuous variable of interest. As each voxel belongs to multiple overlapping neighborhoods, the statistic for a given voxel is determined by combining weights from all neighborhoods to which the voxel participates. Finally, nonparametric permutation testing is used to obtain a voxelwise significance map. Using both simulated and real fMRI data, we demonstrate the effectiveness of the proposed method.

Dimensionality reduction for microarray data using local mean based discriminant analysis

A new method is proposed for finding a low dimensional subspace of high dimensional microarray data. We developed a new criterion for constructing the weight matrix by using local neighborhood information to discover the intrinsic discriminant structure in the data. Also this approach applies regularized least square technique to extract relevant features. We assess the performance of the proposed methodology by applying it to four publicly available tumor datasets. In a low dimensional subspace, the proposed method classified these tumors accurately and reliably. Also, through a comparison study, we verify the reliability of the dimensionality reduction and discrimination results.

Variable Regularized Square Root Recursive Least Square Method

In this paper, we develop the variable regularized square root recursive least square method which is in order to track time-varying parameters extended by an exponential forgetting factor (EF-VR-SRRLS). The proposed approach is arisen from the exact recursification of the original batch mode method solving the restricted quadratic problem. The meanings of the restricted conditions lie in the removal of ambiguity of the cost function and thus to ensure the regularity of the direction matrix. In the examined approach, the regularization terms are the matrix weighting the difference between the next parameter estimate and the initial parameter estimate and the time-varying matrix weighting the distance of the parameter estimate from its currently available estimate. Because the incorporation of the Potter's square root technique requires the actualization of direction matrix using the data dyads only, an effective form of implementation based on the updating quantity of the rank one was performed. The performance of the suggested algorithm combining variable regularization and the square root filtering technique is verified via simulation studies.

Scattered data reconstruction by regularization in B-spline and associated wavelet spaces

The problem of fitting a nice curve or surface to scattered, possibly noisy, data arises in many applications in science and engineering. In this paper we solve the problem using a standard regularized least square framework in an approximation space spanned by the shifts and dilates of a single compactly supported function ϕ . We first provide an error analysis to our approach which, roughly speaking, states that the error between the exact (probably unknown) data function and the obtained fitting function is small whenever the scattered samples have a high sampling density and a low noise level. We then give a computational formulation in the univariate case when ϕ is a uniform B-spline and in the bivariate case when ϕ is the tensor product of uniform B-splines. Though sparse, the arising system of linear equations is ill-conditioned; however, when written in terms of a short support wavelet basis with a well-chosen normalization, the resulting system, which is symmetric positive definite, appears to be well-conditioned, as evidenced by the fast convergence of the conjugate gradient iteration. Finally, our method is compared with the classical cubic/thin-plate smoothing spline methods via numerical experiments, where it is seen that the quality of the obtained fitting function is very much equivalent to that of the classical methods, but our method offers advantages in terms of numerical efficiency. We expect that our method remains numerically feasible even when the number of samples in the given data is very large.

Diffusion regularization for iterative reconstruction in emission tomography

We have recently proposed a regularized least square criterion for adaptive regularization of single photon emission computed tomography (SPECT) reconstruction with nonuniform attenuation correction. In the present study, we show that this regularization is closely related to a diffusion scheme used for Gaussian filtering. For a given value of the regularization parameter, the amount of smoothing is independent from the patient attenuation map, and it is mathematically related to the full-width at half-maximum (FWHM) of a Gaussian filter. A second regularized least square criterion is then derived for which regularization also behaves as a diffusion scheme. The new penalty is then shown to be also applicable to the weighted least square criterion, and to the Poisson maximum-likelihood criterion for positron emission tomography (PET) data (i.e., without attenuation) solved by the expectation maximization (EM) algorithm. For all these criteria, the regularization level can thus be set as the FWHM of a Gaussian filter.

Diffusion regularization for iterative reconstruction in emission tomography

We have recently proposed a regularized least square criterion for adaptive regularization of SPECT reconstruction with nonuniform attenuation correction. In the present study, we show that this regularization is closely related to a diffusion scheme used for Gaussian filtering. For a given value of the regularization parameter, the amount of smoothing is independent from the patient attenuation map, and it is mathematically related to the full width at half maximum (FWHM) of a Gaussian filter. A second regularized least square criterion is then derived for which regularization also behaves as a diffusion scheme. The new penalty is then shown to be also applicable to the weighted least square criterion, and to the Poisson maximum likelihood criterion for PET data (i.e., without attenuation) solved by the EM algorithm. For all these criteria, the regularization level can thus be set as the FWHM of a Gaussian filter.