Error Entropy Research Articles

Design and Analysis of Reversible Data Hiding Using Hybrid Cryptographic and Steganographic approaches for Multiple Images

Data concealing is the process of including some helpful information on images. The majority of sensitive applications, such sending authentication data, benefit from data hiding. Reversible data hiding (RDH), also known as invertible or lossless data hiding in the field of signal processing, has been the subject of a lot of study. A piece of data that may be recovered from an image to disclose the original image is inserted into the image during the RDH process to generate a watermarked image. Lossless data hiding is being investigated as a strong and popular way to protect copyright in many sensitive applications, such as law enforcement, medical diagnostics, and remote sensing. Visible and invisible watermarking are the two types of watermarking algorithms. The watermark must be bold and clearly apparent in order to be visible. To be utilized for invisible watermarking, the watermark must be robust and visibly transparent. Reversible data hiding (RDH) creates a marked signal by encoding a piece of data into the host signal. Once the embedded data has been recovered, the original signal may be accurately retrieved. For photos shot in poor illumination, visual quality is more important than a high PSNR number. The DH method increases the contrast of the host picture while maintaining a high PSNR value. Histogram equalization may also be done concurrently by repeating the embedding process in order to relocate the top two bins in the input image's histogram for data embedding. It's critical to assess the images after data concealment to see how much the contrast has increased. Common picture quality assessments include peak signal to noise ratio (PSNR), relative structural similarity (RSS), relative mean brightness error (RMBE), relative entropy error (REE), relative contrast error (RCE), and global contrast factor (GCF). The main objective of this paper is to investigate the various quantitative metrics for evaluating contrast enhancement. The results show that the visual quality may be preserved by including a sufficient number of message bits in the input photographs.

Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion.

The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems.

Hyperparameter Free MEEF-Based Learning for Next Generation Communication Systems

Information theoretic learning (ITL) criteria have emerged useful for mitigating degradations caused by unknown non-Gaussian noise processes in future wireless communication systems. Specifically, the reproducing kernel Hilbert space (RKHS) based approaches relying on ITL based learning criteria are envisioned to provide near-optimal mitigation of unknown hardware impairments and non-Gaussian noises. Among several ITL criteria, the recent works find the minimum error entropy with fiducial points (MEEF) promising due to its guarantee of unbiased estimation and generalization over generic noise distributions. However, MEEF based learning approaches are known to depend on an accurate kernel-width initialization. Also, the optimal value of this kernel-width is well-known to vary temporally and across deployment scenarios. To remove the dependency on kernel-width, a hyperparameter-free MEEF based adaptive algorithm is derived using random-Fourier features with sampled kernel widths (RFF-SKW). In addition, a detailed convergence analysis is presented for the proposed hyperparameter-free MEEF, which promises a near-optimal error-floor independent of step-size and guarantees convergence for a wide range of step sizes. The promised hyperparameter-independence and improved convergence for the proposed hyperparameter-free MEEF are validated by computer simulations considering different case studies.

Cubature Kalman Filter Under Minimum Error Entropy With Fiducial Points for INS/GPS Integration

Traditional cubature Kalman filter (CKF) is a preferable tool for the inertial navigation system (INS)/global positioning system (GPS) integration under Gaussian noises. The CKF, however, may provide a significantly biased estimate when the INS/GPS system suffers from complex non-Gaussian disturbances. To address this issue, a robust nonlinear Kalman filter referred to as cubature Kalman filter under minimum error entropy with fiducial points (MEEF-CKF) is proposed. The MEEF-CKF behaves a strong robustness against complex non-Gaussian noises by operating several major steps, i.e., regression model construction, robust state estimation and free parameters optimization. More concretely, a regression model is constructed with the consideration of residual error caused by linearizing a nonlinear function at the first step. The MEEF-CKF is then developed by solving an optimization problem based on minimum error entropy with fiducial points (MEEF) under the framework of the regression model. In the MEEF-CKF, a novel optimization approach is provided for the purpose of determining free parameters adaptively. In addition, the computational complexity and convergence analyses of the MEEF-CKF are conducted for demonstrating the calculational burden and convergence characteristic. The enhanced robustness of the MEEF-CKF is demonstrated by Monte Carlo simulations on the application of a target tracking with INS/GPS integration under complex non-Gaussian noises.

Multi-sensor measurement fusion based on minimum mixture error entropy with non-Gaussian measurement noise

In this paper, a novel multi-sensor measurement fusion method is proposed based on minimum error entropy criterion for nonlinear state estimation in non-Gaussian measurement noise. Through cubature-quadrature transform, a mixture error entropy cost function is defined to capture the high order statistics of multi-source measurement error. Based on the centralized measurement model, a measurement fusion information filter is developed by minimizing the mixture error entropy cost. The fixed point iteration approach is employed to calculate the estimates, and the square root implementation of the proposed filter is also provided. Further, the convergence conditions of the proposed information filter are derived. Simulations are performed with different dimensional measurements to demonstrate the effectiveness of the proposed algorithm. It is shown that, the estimation performance of the proposed measurement fusion information filter is better than that of traditional Kalman and Huber's filter based fusion methods against outliers and Gaussian mixture noises.

Transmission Line Parameter Estimation Under Non-Gaussian Measurement Noise

Accurate knowledge of transmission line parameters is essential for a variety of power system monitoring, protection, and control applications. The use of phasor measurement unit (PMU) data for transmission line parameter estimation (TLPE) is well-documented. However, existing literature on PMU-based TLPE implicitly assumes the measurement noise to be Gaussian. Recently, it has been shown that the noise in PMU measurements (especially in the current phasors) is better represented by Gaussian mixture models (GMMs), i.e., the noises are non-Gaussian. We present a novel approach for TLPE that can handle non-Gaussian noise in the PMU measurements. The measurement noise is expressed as a GMM, whose components are identified using the expectation-maximization (EM) algorithm. Subsequently, noise and parameter estimation is carried out by solving a maximum likelihood estimation problem iteratively until convergence. The superior performance of the proposed approach over traditional approaches such as least squares and total least squares as well as the more recently proposed minimum total error entropy approach is demonstrated by performing simulations using the IEEE 118-bus system as well as proprietary PMU data obtained from a U.S. power utility.

Centered Error Entropy-Based Sigma-Point Kalman Filter for Spacecraft State Estimation with Non-Gaussian Noise

The classical sigma-point Kalman filter (SPKF) is widely used in a spacecraft state estimation area with the Gaussian white noise hypothesis. The actual sensor noise is often disturbed by outliers in the harsh space environment, and the SPKF algorithm will reduce the filtering accuracy or even diverge. In this study, to enhance the robustness under non-Gaussian noise condition, the outlier-robust SPKF algorithm based on a centered error entropy (CEE) criterion is derived. Unscented Kalman filter (UKF) is typical of SPKF; combining the deterministic sampling criterion with the centered error entropy criterion, a robust centered error entropy UKF (CEEUKF) algorithm is proposed. The CEEUKF uses the unscented transformation (UT) method to perform time update step and then uses the robust regression model and CEE criterion to reconstruct the measurement update step. The effectiveness of the proposed CEEUKF is verified by a spacecraft attitude determination system.

On Quantization Errors in Approximate and Sample Entropy.

Approximate and sample entropies are acclaimed tools for quantifying the regularity and unpredictability of time series. This paper analyses the causes of their inconsistencies. It is shown that the major problem is a coarse quantization of matching probabilities, causing a large error between their estimated and true values. Error distribution is symmetric, so in sample entropy, where matching probabilities are directly summed, errors cancel each other. In approximate entropy, errors are accumulating, as sums involve logarithms of matching probabilities. Increasing the time series length increases the number of quantization levels, and errors in entropy disappear both in approximate and in sample entropies. The distribution of time series also affects the errors. If it is asymmetric, the matching probabilities are asymmetric as well, so the matching probability errors cease to be mutually canceled and cause a persistent entropy error. Despite the accepted opinion, the influence of self-matching is marginal as it just shifts the error distribution along the error axis by the matching probability quant. Artificial lengthening the time series by interpolation, on the other hand, induces large error as interpolated samples are statistically dependent and destroy the level of unpredictability that is inherent to the original signal.

Mixture quantized error entropy for recursive least squares adaptive filtering

Error entropy is a well-known learning criterion in information theoretic learning (ITL), and it has been successfully applied in robust signal processing and machine learning. To date, many robust learning algorithms have been devised based on the minimum error entropy (MEE) criterion, and the Gaussian kernel function is always utilized as the default kernel function in these algorithms, which is not always the best option. To further improve learning performance, two concepts using a mixture of two Gaussian functions as kernel functions, called mixture error entropy and mixture quantized error entropy, are proposed in this paper. We further propose two new recursive least-squares algorithms based on mixture minimum error entropy (MMEE) and mixture quantized minimum error entropy (MQMEE) optimization criteria. The convergence analysis, steady-state mean-square performance, and computational complexity of the two proposed algorithms are investigated. In addition, the reason why the mixture mechanism (mixture correntropy and mixture error entropy) can improve the performance of adaptive filtering algorithms is explained. Simulation results show that the proposed new recursive least-squares algorithms outperform other RLS-type algorithms, and the practicality of the proposed algorithms is verified by the electro-encephalography application.

A kernel recursive minimum error entropy adaptive filter

The minimum error entropy, a currently useful alternative criterion, is widely adopted in the signal processing domain against impulsive noise. In this brief, we propose a novel algorithm to blend the advantages of both the kernel recursive least squares algorithm and the minimum error entropy criterion, called kernel recursive minimum error entropy algorithm. The proposed new algorithm achieves better recovery performance in predicting the Mackey–Glass time series, equalizing the nonlinear channel under heavy tailed alpha-stable environments and processing EEG data.

Identification of an ARX model with impulse noise using a variable step size information gradient algorithm based on the kurtosis and minimum Renyi error entropy

AbstractARX model is an autoregressive model with exogenous terms. Because of its simplicity and easy parameterization, the ARX model has been widely used in various applications. However, most reports on ARX identification are about Gaussian noise or white noise environment. In many practical industrial applications, impulse noise widely exists. For systems contaminated by this noise, the performance of the mean square error algorithm will deteriorate. To get more accurate results, a variable step size stochastic information gradient algorithm is proposed. The algorithm is based on the Renyi square error entropy and introduces a fourth‐order statistic of the error–kurtosis–into the variable step size, which not only effectively suppresses the impulse noise, but also accelerates the convergence speed. At the same time, a simple method of determining the maximum step size is given. The computational cost and the convergence are also analyzed. Numerical experiments and case study show that for the ARX model disturbed by impulse noise, the proposed algorithm can obtain high‐precision parameter estimates with fast convergence speed.

Prediction of Ship Heave Motion Using Regularized BP Neural Network with Cross Entropy Error Function

Accurate prediction of ship’s heave motion can greatly enhance the safety of offshore operation. Due to its complexity and nonlinearity, however, ship’s heave motion prediction is a difficult task to be solved. In this paper, a new method for predicting ship’s heave motion is proposed based on an improved back propagation neural network (IBPNN). To overcome the gradient saturation phenomenon of traditional BPNN, the mean square error (MSE) loss function is replaced with a cross entropy (CE) loss function in IBPNN. Meanwhile, the weights of IBPNN is regularized by L_2 norm to enhance the generalization ability of traditional BPNN. Finally, conjugate gradient method is adopted to train IBPNN. The IBPNN is used to predict ship’s heave motion and the prediction results prove its effectiveness.

Methods of Estimating the Form of the Probability Distribution Density in Tasks of Processing Measurement Results

Issues associated with methods for estimating the shape of the probability distribution density curve are analyzed in order to classify them when processing measurement results. For example, such nonparametric methods as the method of histograms and frequency polygon, as well as the method of classification of distributions, are considered. It is shown that the values of the anticurtosis and entropy coefficient can be taken as independent features of the form of symmetric distributions. For probability distribution densities that have a one-sided character, such as multiplicative noise, a skewness coefficient should be added to the parameters to consider. Recurrent procedures for obtaining current estimates of numerical characteristics of analyzed random processes are given. The results of processing a random process based on recurrent procedures are presented. It is shown that when the number of samples increases, the estimates obtained by using recurrent and non-recurrent procedures converge. The scattering of estimates of probability distribution density parameters, such as variance, relative mean square error, and entropy error, is determined.

Bayesian Estimations of Exponential Distribution Based on Interval-Censored Data with a Cure Fraction

Censored data are considered to be of the interval type where the upper and lower bounds of an event’s failure time cannot be directly observed but only determined between interval inspection times. The analyses of interval-censored data have attracted attention because they are common in the fields of reliability and medicine. A proportion of patients enrolled in clinical trials can sometimes be cured. In some instances, their symptoms mostly disappear without any recurrence of the disease. In this study, the proportion of such patients who are cured is estimated. Furthermore, the Bayesian approach under the gamma prior and maximum likelihood estimation (MLE) is used to estimate the cure fraction depending on the bounded cumulative hazard (BCH) model based on interval-censored data with an exponential distribution. The Bayesian approach uses three loss functions: squared error, linear exponential, and general entropy. These functions are compared with the MLE and used between estimators. Moreover, they are obtained using the mean squared error, which locates the best option to estimate the parameter of an exponential distribution. The results show that the BCH model and lambda parameter of the exponential distribution based on the interval-censored data can be best estimated using the Bayesian gamma prior with a positive loss function of the linear exponential.

Machine learning based decipherment of Cell Population Data: a promising hospital front-door screening tool for COVID-19

Introduction/ObjectiveKey challenges against early diagnosis of COVID-19 are its symptoms sharing nature and prolong SARS-CoV-2 PCR turnaround time. Hither machine learning (ML) tools experienced by routinely generated clinical data; potentially grant early prediction.Methods/Case ReportRoutine and earlier diagnostic data along demographic information were extracted for total of 21,672 subsequent presentations. Along conventional statistics, multilayer perceptron (MLP) and radial basis function (RBF) were applied to predict COVID-19 from pre-pandemic control. Three feature sets were prepared, and performance evaluated through stratified 10-fold cross validation. With differing predominance of COVID-19, multiple test sets were created and predictive efficiency was evaluated to simulate real-fashion performance against fluctuating course of pandemic. Models validation was also inducted in prospective manner on independent dataset, equating framework forecasting to conclusions from PCR.Results (if a Case Study enter NA)RBF model attained superior cross entropy error 20.761(7.883) and 20.782(3.991) for Q-Flags and Routine Items respectively while MLP outperformed for cell population data (CPD) parameters with value of 6.968(1.259) for ‘training(testing)’. Our CPD driven MLP framework in challenge of lower (<5%) COVID-19 predominance affords greater negative predictive values (NPV >99%). Higher accuracy (%correct 92.5) was offered during prospective validation using independent dataset. Sensitivity analysis advances illusive accuracy (%correct 94.1) and NPV (96.9%). LY-WZ, Blasts/Abn Lympho?, ‘HGB Interf?’, and ‘RBC Agglutination?’ are among novel enlightening study attributes.ConclusionCPD driven ML tools offer efficient screening of COVID-19 patients at presentation to hospital to backing early expulsion and directing patients’ flow-from amid the initial presentation to hospital.

Automated system for identification of data distribution laws by analysis of histogram proximity with sample reduction

The error in the identification of the distribution law entails an incorrect assessment of other characteristics (standard deviation, kurtosis, antikurtosis, etc.). The article is devoted to the development of accessible and simple software products for solving problems of identifying distribution laws and determining the optimal size of a data sample.&#x0D; The paper describes a modified method for identifying the law of data distribution by visual analysis of the proximity of histograms with a reduction in the sample size with software implementation. The method allows choosing the most probable distribution law from a wide base of the set. The essence of the method consists in calculating the entropy coefficient and absolute entropy error for the initial and half data sample, determining the optimal method for processing the histogram using visual analysis of the proximity of histograms, and identifying the data distribution law. The experimental data processing model makes it possible to take into account the statistical properties of real data and can be applied to various arrays, and allows to reduce the sample size required for analysis.&#x0D; An automated system for identifying the laws of data distribution with a simple and intuitive interface has been developed. The results of the study on real data indicate an increase in the reliability of the identification of the data distribution law.

A combined generalized Warblet transform and second order synchroextracting transform for analyzing nonstationary signals of rotating machinery

In recent years, considerable attention has been paid in time–frequency analysis (TFA) methods, which is an effective technology in processing the vibration signal of rotating machinery. However, TFA techniques are not sufficient to handle signals having a strong non-stationary characteristic. To overcome this drawback, taking short-time Fourier transform as a link, a TFA methods that using the generalized Warblet transform (GWT) in combination with the second order synchroextracting transform (SSET) is proposed in this study. Firstly, based on the GWT and SSET theories, this paper proposes a method combining the two TFA methods to improve the TFA concentration, named GWT–SSET. Secondly, the method is verified numerically with single-component and multi-component signals, respectively. Quantized indicators, Rényi entropy and mean relative error (MRE) are used to analyze the concentration of TFA and accuracy of instantly frequency (IF) estimation, respectively. Finally, the proposed method is applied to analyze nonstationary signals in variable speed. The numerical and experimental results illustrate the effectiveness of the GWT–SSET method.

Minimum error entropy criterion‐based randomised autoencoder

The extreme learning machine-based autoencoder (ELM-AE) has attracted a lot of attention due to its fast learning speed and promising representation capability. However, the existing ELM-AE algorithms only reconstruct the original input and generally ignore the probability distribution of the data. The minimum error entropy (MEE), as an optimal criterion considering the distribution statistics of the data, is robust in handling non-linear systems and non-Gaussian noises. The MEE is equivalent to the minimisation of the Kullback–Leibaler divergence. Inspired by these advantages, a novel randomised AE is proposed by adopting the MEE criterion as the loss function in the ELM-AE (in short, the MEE-RAE) in this study. Instead of solving the output weight by the Moore–Penrose generalised inverse, the optimal output weight is obtained by the fixed-point iteration method. Further, a quantised MEE (QMEE) is applied to reduce the computational complexity of. Simulations have shown that the QMEE-RAE not only achieves superior generalisation performance but is also more robust to non-Gaussian noises than the ELM-AE.

Centered error entropy Kalman filter with application to satellite attitude determination

Due to unavoidable factors, heavy-tailed noise appears in satellite attitude estimation. Traditional Kalman filter is prone to performance degradation and even filtering divergence when facing non-Gaussian noise. The existing robust algorithms have limited accuracy. To improve the attitude determination accuracy under non-Gaussian noise, we use the centered error entropy (CEE) criterion to derive a new filter named centered error entropy Kalman filter (CEEKF). CEEKF is formed by maximizing the CEE cost function. In the CEEKF algorithm, the prior state values are transmitted the same as the classical Kalman filter, and the posterior states are calculated by the fixed-point iteration method. The CEE EKF (CEE-EKF) algorithm is also derived to improve filtering accuracy in the case of the nonlinear system. We also give the convergence conditions of the iteration algorithm and the computational complexity analysis of CEEKF. The results of the two simulation examples validate the robustness of the algorithm we presented.

Application of a multilayer perceptron artificial neural network for identification of peach cultivars based on physical characteristics

In the fresh fruit industry, identification of fruit cultivars and fruit quality is of vital importance. In the current study, nine peach cultivars (Dixon, Early Grande, Flordaprince, Flordastar, Flordaglo, Florda 834, TropicSnow, Desertred, and Swelling) were evaluated for differences in skin color, firmness, and size. Additionally, a multilayer perceptron (MLP) artificial neural network was applied for identification of the cultivars according to these attributes. The MLP was trained with an input layer including six input nodes, a single hidden layer with six hidden nodes, and an output layer with nine output nodes. A hyperbolic tangent activation function was used in the hidden layer and the cross entropy error was given because the softmax activation function was functional to the output layer. Results showed that the cross entropy error was 0.165. The peach identification process was significantly affected by the following variables in order of contribution (normalized importance): polar diameter (100%),L∗(89.0),b∗(88.0%),a∗(78.5%), firmness (71.3%), and cross diameter (37.5.3%). The MLP was found to be a viable method of peach cultivar identification and classification because few identifying attributes were required and an overall classification accuracy of 100% was achieved in the testing phase. Measurements and quantitative discrimination of peach properties are provided in this research; these data may help enhance the processing efficiency and quality of processed peaches.