Minimum Square Error Research Articles

Optimal Model-free Approach Based on MDL and CHL for Active Brain Identification in fMRI Data Analysis.

Cluster analysis is a robust tool for exploring the underlining structures in data and grouping them with similar objects. In the researches of Functional Magnetic Resonance Imaging (fMRI), clustering approaches attempt to classify voxels depending on their time-course signals into a similar hemodynamic response over time. In this work, a novel unsupervised learning approach is proposed that relies on using Enhanced Neural Gas (ENG) algorithm in fMRI data for comparison with Neural Gas (NG) method, which has yet to be utilized for that aim. The ENG algorithm depends on the network structure of the NG and concentrates on an efficacious prototype-based clustering approach. The comparison outcomes on real auditory fMRI data show that ENG outperforms the NG and statistical parametric mapping (SPM) methods due to its insensitivity to the ordering of input data sequence, various initializations for selecting a set of neurons, and the existence of extreme values (outliers). The findings also prove its capability to discover the exact and real values of a cluster number effectively. Four validation indices are applied to evaluate the performance of the proposed ENG method with fMRI and compare it with a clustering approach (NG algorithm) and model-based data analysis (SPM). These validation indices include the Jaccard Coefficient (JC), Receiver Operating Characteristic (ROC), Minimum Description Length (MDL) value, and Minimum Square Error (MSE). The ENG technique can tackle all shortcomings of NG application with fMRI data, identify the active area of the human brain effectively, and determine the locations of the cluster center based on the MDL value during the process of network learning.

SURE-based automatic parameter selection for ESPIRiT calibration.

ESPIRiT is a parallel imaging method that estimates coil sensitivity maps from the auto-calibration region (ACS). This requires choosing several parameters for the optimal map estimation. While fairly robust to these parameter choices, occasionally, poor selection can result in reduced performance. The purpose of this work is to automatically select parameters in ESPIRiT for more robust and consistent performance across a variety of exams. By viewing ESPIRiT as a denoiser, Stein's unbiased risk estimate (SURE) is leveraged to automatically optimize parameter selection in a data-driven manner. The optimum parameters corresponding to the minimum true squared error, minimum SURE as derived from densely sampled, high-resolution, and non-accelerated data and minimum SURE as derived from ACS are compared using simulation experiments. To avoid optimizing the rank of ESPIRiT's auto-calibrating matrix (one of the parameters), a heuristic derived from SURE-based singular value thresholding is also proposed. Simulations show SURE derived from the densely sampled, high-resolution, and non-accelerated data to be an accurate estimator of the true mean squared error, enabling automatic parameter selection. The parameters that minimize SURE as derived from ACS correspond well to the optimal parameters. The soft-threshold heuristic improves computational efficiency while providing similar results to an exhaustive search. In-vivo experiments verify the reliability of this method. Using SURE to determine ESPIRiT parameters allows for automatic parameter selections. In-vivo results are consistent with simulation and theoretical results.

Entropy and mutual information in genome-wide selection: the splitting of k-fold cross-validation sets and implications for tree breeding

Random k-fold cross-validation in genome-wide selection (GWS) can help to estimate predictive ability ($$ {r}_{y\hat{y}} $$). Predictive ability tends to be higher when training, and validation sets present a high degree of kinship. However, many tree breeding populations are less genetically related to the training sets and have different levels of phenotypic diversity. Therefore, this study proposes methods of splitting k-fold cross-validation sets to optimize $$ {r}_{y\hat{y}} $$ estimates that are consistent with the breeding population and verify the impact of phenotypic and genotypic distribution on GWS. Using a simulated Eucalyptus trait (h2=0.5) and Pinus taeda L. data for diameter at breast height (h2=0.31), six methods were developed based on mutual information (I) and entropy (H) for measuring genetic similarity and phenotypic dissimilarity, respectively. All methods were evaluated for $$ {r}_{y\hat{y}} $$, bias, minimum squared error of prediction, and genomic heritability. The Pearson correlations of these parameters with the kinship coefficient, and I and H between and within training and validation sets were also estimated. Our results show that closer genetic similarity did not significantly increase $$ {r}_{y\hat{y}} $$ and that a lower H reduced $$ {r}_{y\hat{y}} $$ and overestimated genomic breeding values. Consequently, phenotypic diversity (high H) should be added to tree breeding populations to increase genetic gain and reduce bias. The new methods accurately fitted models according to the entropy of tree breeding populations and their genetic relationship to the training sets. Therefore, these methods provided usable estimates of genetic gain to produce consistent success of long-term tree breeding programs.

A statistical study on lognormal central tendency estimation in probabilistic seismic assessments

Lognormal distribution is one of the most applicable distributions in many different probabilistic seismic assessments such as seismic hazard analysis, fragility analyses, and reliability evaluations. This study compares the sample median (SM), sample geometric mean (SGM) and Sample Bias Corrected Geometric Mean (SBCGM) as three different estimators of lognormal central estimator from different aspects. To this end, the efficiency of each estimator is investigated by calculating their bias and minimum squared error (MSE) using Monte Carlo Simulations. Further, probabilistic performance of four Steel Moment Resisting Frames (SMRF) is investigated using Incremental Dynamic Analysis (IDA), and the stochastic seismic performance of one of them (5 story frame) is investigated considering the aleatoric and epistemic uncertainties. Additionally, the probabilistic seismic performance of a sample jacket type offshore platform based on the mentioned statistical estimators of lognormal central tendency is investigated implementing the IDA procedure. The results show that the efficiency of SM is significantly lower than the other two estimators. In general, SBCGM is the most efficient one for small sample sizes. However, SBCGM and SGM show the same efficiency as the sample size increases. Also, noticeable differences can be observed in the seismic performance of the structures based on each estimator. According to the results of goodness of fit tests and the likelihood ratio evaluations, SBCGM is recommended for achieving more reliable seismic assessments when using small number of records. As another marginal conclusion, the conducted statistical study revealed that the lognormal assumption is approved for collapse fragility curves considering both aleatoric and epistemic sources of uncertainty.

Cooperative Optimal Controller and Its Application to Activated Sludge Process.

With the increasing complexity and scale of activated sludge process (ASP), it is quite challenging to coordinate the performance indices with different time scales. To address this problem, a cooperative optimal controller (COC) is proposed to improve the operation performance in this paper. First, a cooperative optimal scheme is developed for designing the control system, where the different time-scale performance indices are formulated by two levels. Second, a data-driven surrogate-assisted optimization (DDSAO) algorithm is provided to optimize the cooperative objectives, where a surrogate model is established for evaluating the feasibility of optimal solutions based on the minimum squared error. Third, an adaptive predictive control strategy is investigated to derive the control laws for improving the tracking control performance. Finally, the proposed COC is tested on benchmark simulation model No. 1 (BSM1). The results demonstrate that the proposed COC is able to coordinate the multiple time-scale performance indices and achieve the competitive optimal control performance.

An Efficient ANN Interference Cancelation for High Order Modulation over Rayleigh Fading Channel

High order modulation (HOM) presents a key challenge in increasing spectrum efficiency in 4G and upcoming 5G communication systems. In this paper, two non-linear adaptive equalizer techniques based on multilayer perceptron (MLP) and radial basis function (RBF) are designed and applied on HOM to optimize its performance despite its high sensitivity to noise and channel distortions. The artificial neural network’s (ANN) adaptive equalizer architectures and learning methods are simplified to avoid more complexity and to ensure greater speed in symbol decision making. They will be compared with the following popular adaptive filters: least mean square (LMS) and recursive least squares (RLS), in terms of bit error rate (BER) and minimum square error (MSE) with 16, 64, 128, 256, 512 and 1024 quadrature amplitude modulation (QAM). By that, this work will show the advantage that the MLP equalizer has, in most cases, over RBF and traditional linear equalizers.

MMSE and maximum a posteriori estimators for speech enhancement in additive noise assuming a t ‐location‐scale clean speech prior

The authors derive closed form solutions for the minimum mean square error (MMSE) and maximum a posteriori estimators for speech enhancement in additive Gaussian noise assuming a t -location-scale probability density function (PDF) as clean speech prior. Fitting a t -location-scale PDF to the real and imaginary parts of the discrete fourier transform (DFT) coefficients of clean speech signals demonstrates the lower Jensen–Shannon divergence compared to the other heavy-tailed distributions such as Laplacian and gamma. The authors utilise the two presented estimators along with the Wiener filter and MMSE estimators based on Laplacian, gamma, and generalised gamma prior PDFs to enhance noisy signals from the NOIZEUS database. All the estimators are compared together in terms of both signal and noise distortions. The obtained results show that their proposed MMSE estimator results in the minimum squared error and signal distortion to estimate the complex-valued DFT coefficients of speech. The quality assessments of the enhanced signals are also performed in terms of perceptual evaluation of speech quality, segmental and general SNRs.

Action recognition using restricted dense trajectories

This paper presents an action recognition algorithm using restricted dense trajectories (RDT). In feature extraction step, restricted dense trajectories are obtained by tracking the refined points in optical flow field, which remove most of meaningless trajectories while preserve the discriminative power. Then we extract a new set of descriptors to capture the appearance and motion information of trajectories. For encoding step, we improve VLAD by assigning each descriptor to their K nearest words and employing these words as basis vectors to linearly approximate the descriptor under the minimum squared error criterion. Experimental results show the proposed algorithm obtains state-of-the-art results.

Robust Estimation of Finite Population Totals Using a Model Based Approach in the Presence of Two Auxiliary Variables

The utilization of auxiliary information during surveys increases the accuracy of estimators, thereby giving more reliable estimates of the population parameters of interest. It has been established that the presence of more than one auxiliary variables, some more robust estimators can be formed by combining different estimators like product, ratio or even regression estimators and in each case the individual estimators uses its own random variable. One of the most commonly used methods is the ratio method of estimating finite totals which is the foundation of all the other methods that use auxiliary information. In this paper, an estimator of the ratio-exponential class that uses two auxiliary variables has been proposed and its variance derived. After deriving the proposed estimator the coverage probabilities were estimated. Results showed that the interval length of the proposed estimator was narrower and tighter than that of the known Horwitz-Thompson’s estimator. Two datasets from the agricultural and environmental sectors were used in order to investigate the properties of the estimator and they gave satisfactory results. Mean squared error criteria was used to investigate the performance of the proposed estimator and in both cases it had the minimum squared error values. The analysis in these paper is of very great importance in understanding environmental and agricultural data.

Sparse kernel minimum squared error using Householder transformation and givens rotation

Two obvious limitations exist for baseline kernel minimum squared error (KMSE): lack of sparseness of the solution and the ill-posed problem. Previous sparse methods for KMSE have overcome the second limitation using a regularization strategy, which introduces an increase in the computational cost to determine the regularization parameter. Hence, in this paper, a constructive sparse algorithm for KMSE (CS-KMSE) and its improved version (ICS-KMSE) are proposed which will simultaneously address the two limitations described above. CS-KMSE chooses the training samples that incur the largest reductions on the objective function as the significant nodes on the basis of the Householder transformation. In contrast with CS-KMSE, there is an additional replacement mechanism using Givens rotation in ICS-KMSE, which results in ICS-KMSE giving better performance than CS-KMSE in terms of sparseness. CS-KMSE and ICS-KMSE do not require the regularization parameter at all before they begin to choose significant nodes, which is beneficial since it saves on the model selection time. More importantly, CS-KMSE and ICS-KMSE terminate their procedures with an early stopping strategy that acts as an implicit regularization term, which avoids overfitting and curbs the sparse level on the solution of the baseline KMSE. Finally, in comparison with other algorithms, both ICS-KMSE and CS-KMSE have superior sparseness, and extensive comparisons confirm their effectiveness and feasibility.

Generalization improvement for regularized least squares classification

In the past decades, regularized least squares classification (RLSC) is a commonly used supervised classification method in the machine learning filed because it can be easily resolved through the simple matrix analysis and achieve a close-form solution. Recently, some studies conjecture that the margin distribution is more crucial to the generalization performance. Moreover, from the view of margin distribution, RLSC only considers the first-order statistics (i.e., margin mean) and does not consider the actual higher-order statistics of margin distribution. In this paper, we propose a novel RLSC which takes into account the actual second-order (i.e., variance) information of margin distribution. It is intuitively expected that small margin variance will improve the generalization performance of RLSC from a geometric view. We incorporate the margin variance into the objective function of RLSC and achieve the optimal classifier by minimizing the margin variance. To evaluate the performance of our algorithm, we conduct a series of experiments on several benchmark datasets in comparison with RLSC, kernel minimum squared error, support vector machine and large margin distribution machine. And the empirical results verify the effectiveness of our algorithm and indicate that the margin distribution is helpful to improve the classification performance.

Blood vessel segmentation in color fundus images based on regional and Hessian features.

To propose a new algorithm of blood vessel segmentation based on regional and Hessian features for image analysis in retinal abnormality diagnosis. Firstly, color fundus images from the publicly available database DRIVE were converted from RGB to grayscale. To enhance the contrast of the dark objects (blood vessels) against the background, the dot product of the grayscale image with itself was generated. To rectify the variation in contrast, we used a 5×5 window filter on each pixel. Based on 5 regional features, 1 intensity feature and 2 Hessian features per scale using 9 scales, we extracted a total of 24 features. A linear minimum squared error (LMSE) classifier was trained to classify each pixel into a vessel or non-vessel pixel. The DRIVE dataset provided 20 training and 20 test color fundus images. The proposed algorithm achieves a sensitivity of 72.05% with 94.79% accuracy. Our proposed algorithm achieved higher accuracy (0.9206) at the peripapillary region, where the ocular manifestations in the microvasculature due to glaucoma, central retinal vein occlusion, etc. are most obvious. This supports the proposed algorithm as a strong candidate for automated vessel segmentation.

Biosorption of Pb(II) using Gundelia tournefortii: Kinetics, equilibrium, and thermodynamics

ABSTRACTIn this paper, the feasibility of Gundelia tournefortii was studied as a novel, high-capacity biosorbent for removing lead ions from synthetic wastewater in a batch system. The effects of various parameters such as temperature, initial concentration, initial pH, biosorbent dosage, and contact time were investigated. Based on batch results, the optimum operating conditions were found to be pH 5, biosorbent dosage of 25 mg, and temperature of 20°C in the range of lead initial concentrations from 5 to 100 mg/L. The equilibrium contact time was 60 min. The biosorption mechanism can be well described by the Langmuir isotherm with a monolayer maximum adsorption capacity of 144.928 (mg/g) at 20°C and a pseudo-second-order kinetic model. Thermodynamic studies proved that the sorption process was physical, spontaneous, feasible, random, and exothermic. In the second step, the ability of artificial neural network (ANN) to predict the adsorption capacity of Gundelia tournefortii for the removal of Pb(II) from aqueous solution was examined. The model was developed using a three-layer feed-forward back-propagation (BP) network with 5, 12, and 1 neurons in the first, second, and third layers, respectively. The Levenberg–Marquardt BP training algorithm (LMA) was found to be the best BP algorithm with a minimum mean squared error of 0.000867 and a minimum relative squared error of 0.032771. The comparison between the results of ANN and experimental data showed that ANN has a superior performance (R2= of 0.998) in the prediction of the Pb(II) removal process.

A study of container ship structures' ultimate strength under corrosion effects

Corrosion takes place in the life cycle of ships, resulting in degradation of structural strengths. Corrosion effects with uncertainty on ultimate strength for container ship structures are studied in this paper. On one hand, the ultimate strength of container ship hull girder subject to combined moments of bending and torsion is studied by Nonlinear Finite Element Analysis (NFEA), and the limit state of failure is derived through Minimum Square Error (MSE) technique based on a series of NFEAs, and the probabilistic characteristics under simultaneous uniform corrosions (however, with different scales) in longitudinal structures are studied for the cases of pure bending, pure torsion and combination of both, which can be a useful reference for the scheduled maintenance of container ships. On the other hand, the ultimate strength of bottom stiffened panel under longitudinal thrust derived from hogging conditions is studied considering effects of key factors such as initial deflection, element sizes, simplified boundary conditions, and lateral seawater pressure etc. Uniform and not simultaneous corrosion across different structures is studied, and surrogate models by Gaussian Process (GP) are built, and the corresponding ultimate strength probabilistic characteristics and its dependency on correlation between different structures’ corrosion are studied.

Does the sensorimotor system minimize prediction error or select the most likely prediction during object lifting?

Using a novel experimental model of object lifting, we tested whether the sensorimotor system models the weight of objects by minimizing lifting errors or by selecting the statistically most likely weight. We found that the sensorimotor system minimizes the square of prediction errors for object lifting. This parallels the results of studies that investigated visually guided reaching, suggesting an overlap in the underlying mechanisms between tasks that involve different sensory systems.

Enhanced manifold regularization for semi-supervised classification.

Manifold regularization (MR) has become one of the most widely used approaches in the semi-supervised learning field. It has shown superiority by exploiting the local manifold structure of both labeled and unlabeled data. The manifold structure is modeled by constructing a Laplacian graph and then incorporated in learning through a smoothness regularization term. Hence the labels of labeled and unlabeled data vary smoothly along the geodesics on the manifold. However, MR has ignored the discriminative ability of the labeled and unlabeled data. To address the problem, we propose an enhanced MR framework for semi-supervised classification in which the local discriminative information of the labeled and unlabeled data is explicitly exploited. To make full use of labeled data, we firstly employ a semi-supervised clustering method to discover the underlying data space structure of the whole dataset. Then we construct a local discrimination graph to model the discriminative information of labeled and unlabeled data according to the discovered intrinsic structure. Therefore, the data points that may be from different clusters, though similar on the manifold, are enforced far away from each other. Finally, the discrimination graph is incorporated into the MR framework. In particular, we utilize semi-supervised fuzzy c-means and Laplacian regularized Kernel minimum squared error for semi-supervised clustering and classification, respectively. Experimental results on several benchmark datasets and face recognition demonstrate the effectiveness of our proposed method.

Performance of sigma-delta quantizers in oversampled odd-stacked CMFB systems

In this paper, we investigate the performance of the joint use of odd-stacked cosine modulated filter banks (CMFBs) and the first- and second-order Sigma-Delta (ΣΔ) quantization for communication systems when the signal expansion frame is infinite. This performance is evaluated in terms of the decrease of the reconstruction error of the signal that is jointly represented through the CMFBs and the ΣΔ quantization schemes. To begin with, we derive closed-form expressions of upper-bounds on the signal reconstruction minimum square error (MSE) for both first- and second-order ΣΔ quantization cases. Such upper-bounds are derived irrespectively of any quantization noise assumption that could be made in the considered ΣΔ quantization scheme. Exploiting the obtained upper bound closed-form expressions, we demonstrate that under a set of conditions, this signal reconstruction MSE decays as $\frac {1}{r^{2}} $ where r denotes the redundancy of the signal expansion frame. The obtained results are shown to be true under the widely used additive white quantization noise assumption, where we determine also explicit analytical signal reconstruction MSE expressions when the CMFBs are combined with first- and second-order quantizers. Simulation results are given to support our claims.

A blind signal‐to‐jamming‐plus‐noise ratio estimation based on second‐order cyclostationarity under typical tone‐jamming and its CRLB

SummaryIn this paper, a blind signal‐to‐jamming‐plus‐noise ratio (SJNR) estimation under typical tone‐jamming is proposed in a Rayleigh fading channel. Based on the differences among communication signal, jamming, and noise in cyclostationarity, the proposed algorithm obtained SJNR values by using different parts of the cyclic autocorrelation matrix of received signals and applying the minimum square error (MSE) method to estimate the communication signal power and jamming‐plus‐noise power, respectively. Through deriving the Cramer–Rao lower bound (CRLB) for the SJNR estimation, the normalized MSE performance is examined. Simulation results show that the proposed estimation performs better than the subspace‐based eigenvalue decomposition estimation on a normalized estimation bias, and its normalized MSE performance is also closer to CRLB. Copyright © 2016 John Wiley & Sons, Ltd.

Design of PIDA Controller Using Bat Algorithm for AVR Power System

In this article, a new proportional, integral, derivative, and acceleration (PIDA) controller for an automatic voltage regulator power system is presented. The controller parameters are designed by considering as an optimization with minimization of square error using bat algorithm. The response of the bat algorithm optimized PIDA (BA-PIDA) controller are observed and compared to the controllers in the literature. The superiority is validated in terms of performance indices (ITAE, IAE and ISE).

A method of combining forward with backward greedy algorithms for sparse approximation to KMSE

The prefitting and backfitting methods are commonly used to sparsify the full solution of naive kernel minimum squared error. As known to us, the forward learning methods including prefitting and backfitting only assist us in finding the suboptimal solutions. To enhance the testing real time further, in this paper by virtue of the idea of incorporating the backward learning algorithm into the forward learning algorithm, two improved schemes on the basis of prefitting and backfitting are proposed. Compared with the original versions, two improved algorithms obtain fewer significant nodes, which indicates much better testing real time. Due to the addition of the backward learning to the forward learning, the proposed algorithms need more training computational costs. Investigations on benchmark data sets and a robot arm example are reported to demonstrate the improved effectiveness.