Squared-error Function Research Articles

Convergence analysis of a novel high order networks model based on entropy error function

It is generally known that the error function is one of the key factors that determine the convergence, stability and generalization ability of neural networks. For most feedforward neural networks, the squared error function is usually chosen as the error function to train the network. However, networks based on the squared error function can lead to slow convergence and easily fall into local optimum in the actual training process. Recent studies have found that, compared to the squared error function, the gradient method based on the entropy error function measures the difference between the probability distribution of the model output and the probability distribution of the true labels during the iterative process, which can be more able to handle the uncertainty in the classification problem, less likely to fall into a local optimum and can learn to converge more rapidly. In this paper, we propose a batch gradient method for Sigma-Pi-Sigma neural networks based on the entropy error function and rigorously demonstrate the weak and strong convergence of the new algorithm in the batch input mode. Finally, the theoretical results and effectiveness of the algorithm are verified by simulation.

Robust Frequency-Domain Timing Offset Estimation for DCO-OFDM Systems

Optical wireless communication (OWC) systems suffer inter-symbol interference (ISI) in time domain due to light emitting diode (LED)’s limited bandwidth at transmitter. As intensity modulation and direct detection (IM/DD) is used, the transmitted signals are unipolar and real-valued rather than bipolar and complex-valued as in radio frequency (RF). Thus, it is challenging to employ the RF-based timing offset estimation for OWC. Also, there is no carrier frequency offset (CFO) like in RF. It is possible to estimate timing offset in frequency domain for two reasons. First, the effect of ISI in time domain is mitigated in frequency domain with correct timing offset estimation, robust against LED’s limited bandwidth. Second, the correlation between frequency-domain signals are high, as signals are bipolar and complex-valued in frequency domain rather than unipolar and real-valued in time domain. Therefore, we propose an effective frequency-domain timing offset estimation approach for direct current biased optical-orthogonal frequency division multiplexing (DCO-OFDM) OWC systems, where a small number of pilots are applied on few DCO-OFDM subcarriers, leading to very low spectral overhead. The property of no CFO is utilized to formulate an error function, by exploring the correlation between received signals and pilots. We propose a squared error function (SEF)-based timing offset estimation approach, conducted by maximizing the ratio of two successive variances of squared errors. Simulation results show that, with a very low training overhead of 0.48%, the proposed SEF-based approach provides a bit error ratio (BER) performance close to ideal case with perfect timing offset estimation, perfect channel state information and zero-forcing (ZF) equalization.

The Weibull Claim Model: Bivariate Extension, Bayesian, and Maximum Likelihood Estimations

Using a class of claim distributions, we introduce the Weibull claim distribution, which is a new extension of the Weibull distribution with three parameters. The maximum likelihood estimation method is used to estimate the three unknown parameters, and the asymptotic confidence intervals and bootstrap confidence intervals are constructed. In addition, we obtained the Bayesian estimates of the unknown parameters of the Weibull claim distribution under the squared error and linear exponential function (LINEX) and the general entropy loss function. Since the Bayes estimators cannot be obtained in closed form, we compute the approximate Bayes estimates via the Markov Chain Monte Carlo (MCMC) procedure. By analyzing the two data sets, the applicability and capabilities of the Weibull claim model are illustrated. The fatigue life of a particular type of Kevlar epoxy strand subjected to a fixed continuous load at a pressure level of 90% until the strand fails data set was analyzed.

Chaotic-based particle swarm optimization algorithm for optimal PID tuning in automatic voltage regulator systems

Introduction. In an electrical power system, the output of the synchronous generators varies due to disturbances or sudden load changes. These variations in output severely affect power system stability and power quality. The synchronous generator is equipped with an automatic voltage regulator to maintain its terminal voltage at rated voltage. Several control techniques utilized to improve the response of the automatic voltage regulator system, however, proportional integral derivative (PID) controller is the most frequently used controller but its parameters require optimization. Novelty. In this paper, the chaotic sequence based on the logistic map is hybridized with particle swarm optimization to find the optimal parameters of the PID for the automatic voltage regulator system. The logistic map chaotic sequence-based initialization and global best selection enable the algorithm to escape from local minima stagnation and improve its convergence rate resulting in best optimal parameters. Purpose. The main objective of the proposed approach is to improve the transient response of the automatic voltage regulator system by minimizing the maximum overshoot, settling time, rise time, and peak time values of the terminal voltage, and eliminating the steady-state error. Methods. In the process of parameter tuning, the Chaotic particle swarm optimization technique was run several times through the proposed hybrid objective function, which accommodates the advantages of the two most commonly used objective functions with a minimum number of iterations, and an optimal PID gain value was found. The proposed algorithm is compared with current metaheuristic algorithms including conventional particle swarm optimization, improved kidney algorithm, and others. Results. For performance evaluation, the characteristics of the integral of time multiplied squared error and Zwe-Lee Gaing objective functions are combined. Furthermore, the time-domain analysis, frequency-domain analysis, and robustness analysis are carried out to show the better performance of the proposed algorithm. The result shows that automatic voltage regulator tuned with the chaotic particle swarm optimization based PID yield improvement in overshoot, settling time, and function value of 14.41 %, 37.91 %, 1.73 % over recently proposed IKA, and 43.55 %, 44.5 %, 16.67 % over conventional particle swarm optimization algorithms. The improvement in transient response further improves the automatic voltage regulator system stability for electrical power systems.

Least-squares solutions to polynomial systems of equations with quantum annealing

This work proposes and analyzes a methodology for finding least-squares solutions to the systems of polynomial equations. Systems of polynomial equations are ubiquitous in computational science, with major applications in machine learning and computer security (i.e., model fitting and integer factorization). The proposed methodology maps the squared-error function for a polynomial equation onto the Ising–Hamiltonian model, ensuring that the approximate solutions (by least squares) to real-world problems can be computed on a quantum annealer even when the exact solutions do not exist. Hamiltonians for integer factorization and polynomial systems of equations are implemented and analyzed for both logical optimality and physical practicality on modern quantum annealing hardware.

Estimating Two Parameters of Lomax Distribution by Using the Upper Recorded Values under Two Balanced Loss Functions

In this paper, two lomax distribution parameters are estimated along with the estimation of the reliability function under two balanced loss functions: the balanced squared error function and balanced linex loss function. These two functions depend on both Bayesian and maximum likelihood estimators using one type of generalized order statistics, which is the upper recorded values. The simulation approach using matlab language program is adopted in order to generate the data and compute the estimators. The comparison between shape parameter ( ) estimation methods is done by using posterior Bayesian risk function. The findings show that the estimators under two balanced loss functions are more efficient than the estimators under the two ordinary loss functions .

Rice and wheat straw ashes: Characterization and modeling of pretilachlor sorption kinetics and adsorption isotherm

The rice straw ash (RSA) and wheat straw ash (WSA) were explored as low cost adsorbent for pretilachlor removal from water. The ashes were characterized and sorption behavior of pretilachlor was evaluated. Kinetics study suggested that the modified Elovich model best explained the pretilachlor sorption on both the ashes. The adsorption data were analyzed using 2-, 3- and 4-parameter models and nine error functions were used to compute the best fit isotherm by nonlinear regression analysis. The pretilachlor was more sorbed onto the RSA (22.0–92.2%) than the WSA (11.4–61.4%) and percent adsorption decreased with increase in the herbicide concentration in solution. Isotherm model optimization analysis suggested that the Freundlich and the Temkin isotherms were the best models to predict the pretilachlor adsorption onto the RSA and the WSA. The error analysis suggested that the reciprocal of the observed squared (ROS) and the reciprocal of the predicted squared (RPS) error functions provided the best determination of the adsorption constants for the Freundlich and the Temkin isotherms, respectively. The RSA, which exhibited higher pretilachlor sorption potential, can be utilized as low cost adsorbent for pesticide removal from contaminated water.

Robust TOA source localisation algorithm using prior location

This study considers robust time-of-arrival (TOA) source localisation algorithms. Range measurements are used to estimate the location parameter for TOA source localisation and previous information on the position of the source is employed to improve the existing measurement-based method. The proposed methods are categorised into two types. First, the mean square error between the true parameter and estimate is minimised. Second, the cost function where the penalty function is combined with the squared error function between the measurement and estimate is minimised. The positioning accuracy of the proposed algorithm is much higher than that of the conventional methods in regimes of large noise variances.

Optimal Tuning of Fractional Order PID Controller for DC Motor Speed Control via Chaotic Atom Search Optimization Algorithm

In this paper, atom search optimization (ASO) algorithm and a novel chaotic version of it [chaotic ASO (ChASO)] are proposed to determine the optimal parameters of the fractional-order proportional+integral+derivative (FOPID) controller for dc motor speed control. The ASO algorithm is simple and easy to implement, which mathematically models and mimics the atomic motion model in nature, and is developed to address a diverse set of optimization problems. The proposed ChASO algorithm, on the other hand, is based on logistic map chaotic sequences, which makes the original algorithm be able to escape from local minima stagnation and improve its convergence rate and resulting precision. First, the proposed ChASO algorithm is applied to six unimodal and multimodal benchmark optimization problems and the results are compared with other algorithms. Second, the proposed ChASO-FOPID, ASO-FOPID, and ASO-PID controllers are compared with GWO-FOPID, GWO-PID, IWO-PID, and SFS-PID controllers using the integral of time multiplied absolute error (ITAE) objective function for a fair comparison. Comparisons were also made for the integral of time multiplied squared error (ITSE) and Zwe-Lee Gaing's (ZLG) objective function as the most commonly used objective functions in the literature. Transient response analysis, frequency response (Bode) analysis, and robustness analysis were all carried out. The simulation results are promising and validate the effectiveness of the proposed approaches. The numerical simulations of the proposed ChASO-FOPID and ASO-FOPID controllers for the dc motor speed control system demonstrated the superior performance of both the chaotic ASO and the original ASO, respectively.

Optimization of Microchannel Heat Sinks Using Prey-Predator Algorithm and Artificial Neural Networks

A rectangular microchannel heat sink is modeled by employing thermal resistance and pressure drop networks. The available correlations for both thermal resistance and pressure drop are utilized in optimization. A multi-objective optimization technique, the prey–predator algorithm, is employed with the objective to find the optimal values for the heat sink performance parameters, i.e., thermal resistance and the pumping power of the heat sink. Additionally, a radial basis function neural network is used to investigate a relationship between these parameters. Full training based on the prey–predator algorithm with the sum of the squared error function is used to achieve the best performance of the model. The analysis of variance method is also employed to test the performance of this model. This study shows that the multi-objective function based on the prey–predator algorithm and the neural networks is suitable for finding the optimal values for the microchannel heat sink parameters. The minimum values of the multi-objective function are found to be “pumping power = 2.79344” and “total thermal resistance = 0.134133”.

A class of improved informative priors for bayesian analysis of two-component mixture of failure time distributions from doubly censored data

The Bayesian analysis of the mixture models has received a sizable attention of the analysts during recent years. However, most of the contributions have been discussed under singly type I censored samples. This paper aims to discuss the Bayesian analysis of the two-component mixture of lifetime distribution, with a particular case for Rayleigh distribution, under doubly censored samples. A class of improved informative priors has been assumed for posterior estimation. The squared error and k-loss functions have been proposed to derive the Bayes estimators and the corresponding posterior risks. The prior elicitation has been discussed via prior predictive approach. The comparisons among the performance of different estimators have been made in terms of posterior risks based on analysis of simulated and real life data sets.

A hybrid collaborative filtering model for social influence prediction in event-based social networks

Event-based social networks (EBSNs) provide convenient online platforms for users to organize, attend and share social events. Understanding users’ social influences in social networks can benefit many applications, such as social recommendation and social marketing. In this paper, we focus on the problem of predicting users’ social influences on upcoming events in EBSNs. We formulate this prediction problem as the estimation of unobserved entries of the constructed user-event social influence matrix, where each entry represents the influence value of a user on an event. In particular, we define a user's social influence on a given event as the proportion of the user's friends who are influenced by him/her to attend the event. To solve this problem, we present a hybrid collaborative filtering model, namely, Matrix Factorization with Event-User Neighborhood (MF-EUN) model, by incorporating both event-based and user-based neighborhood methods into matrix factorization. Due to the fact that the constructed social influence matrix is very sparse and the overlap values in the matrix are few, it is challenging to find reliable similar neighbors using the widely adopted similarity measures (e.g., Pearson correlation and Cosine similarity). To address this challenge, we propose an additional information based neighborhood discovery (AID) method by considering both event-specific and user-specific features in EBSNs. The parameters of our MF-EUN model are determined by minimizing the associated regularized squared error function through stochastic gradient descent. We conduct a comprehensive performance evaluation on real-world datasets collected from DoubanEvent. Experimental results show that our proposed hybrid collaborative filtering model is superior than several alternatives, which provides excellent performance with RMSE and MAE reaching 0.248 and 0.1266 respectively in the 90% training data of 10000 users dataset.

Robust collaborative filtering based on non-negative matrix factorization and R1-norm

Collaborative filtering systems are vulnerable to shilling attacks or profile injection attacks in which malicious users can deliberately manipulate the systems’ recommendation output by inserting a number of fake profiles. To address this issue, some robust collaborative filtering methods based on matrix factorization have been proposed. However, these methods suffer from low robustness and recommendation accuracy because they use the squared error function as the loss function that can be easily dominated by large residuals. In this paper we propose a robust collaborative filtering method based on non-negative matrix factorization and R1-norm. Firstly, we introduce R1-norm to construct a robust non-negative matrix factorization model for collaborative filtering and make an analysis of the stability of the model. Secondly, we propose an iterative optimization method of feature matrices based on the iterative updating algorithm of non-negative matrix factorization, which guarantees that the predicted ratings are accurate and non-negative. Finally, we devise a robust collaborative filtering algorithm based on the proposed model. Experimental results on two different datasets show that the proposed method can improve the robustness and recommendation accuracy.

An Intelligent Approach to Educational Data: Performance Comparison of the Multilayer Perceptron and the Radial Basis Function Artificial Neural Networks

(ProQuest: ... denotes formulae omitted)An intelligence system basically consists of three layers (i.e., input, output, and hidden), which contain built-in neurons that connect one layer to another. In a neural network, basic functions are inferred from the data, which allow the network to understand complex interactions between predictor variables (Gonzalez-Camacho et al., 2012; Hastie, Tibshirani, & Friedman, 2009). Artificial neural networks (ANNs) are computational models based on parallel distributed processing, which can be used to model highly complex and non-linear stochastic problems such as the ability to learn, generalize, classify, and organize data (Gomes & Awruch, 2004; Sharaf, Noureldin, Osman, & El-Sheimy, 2005). ANNs can also be used to analyze complex data structures and large data sets, and these so-called intelligence systems are capable of generalizing the findings of scientific studies (Santos, Rupp, Bonzi, & Filed, 2013). Inspired by the thinking patterns of the human brain, ANNs can learn the data structures and conduct numerous statistical processes such as parameter estimations, classifications, and optimizations. In other words, learning in ANNs is accomplished through algorithms that mimic the learning mechanisms of biological systems (Yilmaz & Ozer, 2009). Therefore, the present study investigates the factors that affect the success of university students by employing two artificial neural network methods (i.e., multilayer perceptron and radial basis function), and compares the effects of these methods on educational data in terms of predictive ability.Multilayer Perceptron Artificial Neural NetworkThe multilayer perceptron artificial neural network (MTPANN) and the radial basis function artificial neural network (RBFANN) are both widely used as supervised training methods. Although their structures are somewhat similar, the RBFANN is used to solve scientific problems, whereas the MTPANN is applied for pattern recognition or classification problems by using the error back propagation algorithm. The main purpose of this algorithm is to minimize estimation error by computing all of the weights in the network. In addition, this algorithm systematically updates these weights in order to achieve the best neural network configuration. Essentially, this algorithm consists of two steps: propagation and weight update. Basically, the propagation step involves forward propagations (producing output activations) and backward ones (computing the difference between input (X ) and output (Y) by using output activations). In the weight update process, synaptic weight is multiplied by the delta (X.-Y.) to obtain the gradient weight. Then, a percentage of the gradient weight is subtracted to obtain the rate of pattern recognition. In this case, if the percentage is low, then the accuracy of the training is high. Moreover, if the percentage is high, then the training of the neurons is faster. During this process, the two steps (propagation and weight update) are repeated until the performance of the network architecture is satisfactory.Back propagation needs to compute the derivative of the squared error function by considering the weights in the network. Assuming one output neuron, the squared error function can be computed as follows:...(1)where t is the target output, y is the actual output, and E is the squared error. Each output (O ) that is matched with each neuron can be expressed as follows:...(2)The net. (input) to a neuron is the weighted sum of output O,. In addition, w is the weight between neurons and j. In general, the activation function of the hidden and output layers is non-linear and differentiable. A common activation function is shown as follows:...(3)The activation function in Equation 3 is a logistic form that is commonly used as the activation function. This process continues until the error is optimized. …

Can Critical-Point Paths Under ${\ell}_{{p}}$ -Regularization $(0<p<1)$ Reach the Sparsest Least Squares Solutions?

The solution path of the least square problem under ℓ_p-regularization (0 <; p <; 1) is studied, where the Lagrangian multiplier λ due to the constraint is the parameter of the path. It is first proven that the least square solution of an unconstrained overdetermined linear system is connected with the origin, under a mild condition, by a continuous path of critical points of an ℓ_p-regularized squared error function. Based on this fact, it is proven that every sparsest least square solution of an underdetermined system is connected with the origin by a critical-point path. The existence theorem holds more generally for any least square solution whose support has its associated submatrix of the fat sensing matrix be full column rank. This is a sufficient condition for the existence, and allows to reduce the underdetermined problem to an overdetermined one with the off-support variable(s) nullified. A necessary condition is that the gradient of the ℓ_p regularizer with respect to the support variables lies in the row space of the submatrix (which is not necessarily full column rank).

Robust sparse channel estimation and equalization in impulsive noise using linear programming

In this paper, an algorithm for sparse channel estimation, called ℓ1-regularized least-absolutes (ℓ1-LA), and an algorithm for equalization, called linear least-absolutes (LLA), in non-Gaussian impulsive noise are proposed. The proposed approaches are based on the minimization of the absolute error function, rather than the squared error function. By replacing the standard modulus with the ℓ1-modulus of complex numbers, the resulting optimization problem can be efficiently solved through linear programming. The selection of an appropriate regularization parameter is also addressed. Numerical results demonstrate that the proposed algorithms, compared with the classical methods, are more robust to impulsive noise and have a superior accuracy.

An Efficient Unified K-Means Clustering Technique for Microarray Gene Expression Data

Problem statement: Using microarray techniques one could monitor the expressions levels of thousands of genes simultaneously. One challenge was how to derive meaningful insights into expressed data. This might be carried out by clustering techniques such as hierarchical and k-means, but most of the clustering techniques were largely heuristic in nature and are associated with some unresolved issues like how to fix the precise number of clusters and how to visualize the results in a pictorial form. Approach: Determine accurate number of clusters from gene expression data and validate the results using correctness ratio and sum of squares criteria. A new approach suggested to addresses the primary issue of k-means clustering algorithm that predefining number of clusters. This approach provides accurate number of clusters by minimizing the squared error function and maximizing the correctness ratio value. Results: The experimental results have shown the efficiency of our method by calculating and comparing the sum of squares with different k values. It was concluded that the number of clusters were accurate with minimum sum of squares value and maximum value of correctness ratio. Conclusion: The results showed that the quality of clusters and performance of this new approach is improved.

A note on estimation based on record data

In data-processing standpoint, an efficient algorithm for identifying the minimum value among a set of measurements are record statistics. From a sequence of n independent identically distributed continuous random variables only about log(n) records are expected, so we expect to have little data, hence any prior information is welcome (Houchens, Record value theory and inference, Ph.D. thesis, University of California, Riverside, 1984). In this paper, non-Bayesian and Bayesian estimates are derived for the two parameters of the Exponential distribution based on record statistics with respect to the squared error and Linear-Exponential loss functions and then compared with together. The admissibility of some estimators is discussed.

A Novel Target Field Method for Designing Uniplanar Self-shield Gradient Coils of Fully Open MRI Device

A new approach to the design of uniplanar self-shield gradient coils for fully open magnetic resonance imaging system is proposed to build a specified gradient magnetic field in a target region. In this new design methodology, the surface current density of the coils is represented by a two-dimensional Fourier series expansion. A cost function is constructed in terms of the summation of squared error and penalty functions. Minimizing the cost function, we obtain the continuous current density distribution for the uniplanar gradient coils. Applying the stream function technique to the resulting continuous current distribution, discrete current patterns can be obtained. Employing the Biot-Savart integral equation to the discrete current loops, the gradient magnetic field has been re-evaluated in order to validate the method proposed. Using this approach, uniplanar self-shield x-gradient coils which are capable of creating a gradient field of 40 mT/m have been designed. The results indicate that with this method a magnetic field with high level of linearity and a satisfied screening effect are obtained.

On the Size Corrected Tests in Improved Estimation

In this paper we propose shrinkage preliminary test estimator (SPTE) of the coefficient vector in the multiple linear regression model based on the size corrected Wald ( W), likelihood ratio ( LR) and Lagrangian multiplier ( LM) tests. The correction factors used are those obt,ained from degrees of freedom corrections to the estimate of the error variance and those obtained from the second­order Edgeworth approximations to the exact distributions of the test statistics. The bias and weighted mean squared error (WMSE) fun ctions of the estimators are derived. With respect to WMSE, the relative efficiencies of the SPTEs relative to the maximum likelihood estimator are calculated. This study shows that the amount of conflict can be substantial when the three t ests are based on the same asymptotic chi­square critical value. The conflict among the SPTEs is due to the asymptotic tests not having the correct significance level. The Edgeworth size corrected W, LR and LM tests reduce the conflict remarkably.