Shannon Entropy Criterion Research Articles

Adaptive Line Enhancer for Passive Sonars Based on Frequency-Domain Sparsity, Shannon Entropy Criterion and Mixed-Weighted Error

AbstractAdaptive line enhancer (ALE) is one of the vital signal processing techniques to the detection and recognition of underwater acoustic targets for passive sonars. Conventional ALEs, based on Gaussian noise assumption and least mean square (LMS) algorithm, can achieve good line enhancement property in Gaussian noise background. However, limited by the high steady-state misadjustment of LMS algorithm, the performance of conventional ALEs deteriorates under non-Gaussian noise background and degrades severely in processing signals with comparably lower signal-to-noise ratio (SNR). Therefore, it’s of great necessity to improve the line enhancement performances of ALE techniques to meet the demands of engineering application in passive sonars. In order to optimize the robustness and adaptability of conventional ALEs in dealing with underwater acoustic signals with much lower-SNR and in non-Gaussian noise background, a modified ALE algorithm called frequency-domain ALE based on l1-norm, Shannon entropy criterion and mixed-weighted norm (l1-SE-MWE-FALE) is proposed in this paper. The proposed l1-SE-MWE-FALE algorithm is based on the integration of frequency-domain sparsity, Shannon entropy (SE) criterion along with mixed-weighted error of LMS and least absolute deviation (LAD) to improve the ALE performance in situations above. The simulation results demonstrate that, when the input SNR is as low as – 25 dB, the local SNR (LSNR) gain for line spectrums by l1-SE-MWE-FALE is 9.8 dB, 3.7 dB and 2.3 dB higher than conventional ALE, l1-norm-based frequency-domain ALE (l1-FALE) and l1 norm-Shannon entropy criterion-based frequency-domain ALE (l1-SE-FALE), respectively. Meanwhile, the simulation results also indicate that the parameters of the proposed method can be chosen loosely and hence are insensitive to the choice of their values. Furthermore, the processing results of two different kinds of real ship-radiated noise signals recorded by passive sonars also imply the advantages of the proposed method over the other three ALEs both qualitatively and quantitatively in the respect of line spectrum LSNR gain and parameter insensitivity. The simulation and experiment results both validate the performance insensitivity to parameter adjustment and hence exhibit a good perspective of applications for passive sonars.

Inverse synthetic aperture radar autofocus imaging of block structure targets with sparse aperture

Inverse synthetic aperture radar (ISAR) autofocus imaging performance is challenged by the residual phase errors that arise from the traditional motion compensation methods with sparse aperture. Moreover, sparse reconstruction algorithms usually ignore the block sparsity in the ISAR image, which cannot recover structured information of the block structure target completely. An imaging method based on joint minimum Tsallis entropy and pattern-coupled sparse Bayesian learning algorithm is proposed to achieve ISAR autofocus imaging of block structure targets with sparse aperture. A pattern-coupled hierarchical complex Gaussian prior is utilized to characterize the correlation among ISAR image pixels in the complex domain. Such a prior model has the potential to encourage block-sparse patterns and achieve ISAR sparse aperture imaging of block structure targets without knowing the block partition prior information. The residual phase errors are estimated based on the minimum Tsallis entropy criterion during the sparse reconstruction of ISAR image to achieve ISAR autofocus imaging, which has the advantage of improving the computational efficiency compared with minimum Shannon entropy criterion. The superiority of the proposed algorithm is verified by the experimental results based on both simulated and measured data.

Complex Shannon Entropy Based Learning Algorithm and Its Applications

Nowadays, adaptive filters in complex domain have received increasing attention since they can be applied to a wider scenario than real valued cases. As two typical information theoretic criteria, the minimum complex kernel risk-sensitive loss (MCKRSL) and maximum complex correntropy criterion (MCCC) have shown robustness to non-Gaussian noises in complex domain adaptive filtering. However, since both criteria cannot consider the probability distribution of error, they are not optimal in the presence of some non-Gaussian noises. This paper firstly defines a complex Shannon entropy in terms of the probability distribution of error. Then, a novel adaptive filter is proposed using the minimum complex Shannon entropy (MCSE) criterion. More significantly, the convergence behavior of the MCSE algorithm is discussed, and thus the steady-state excess mean square error (EMSE) is calculated for theoretical analysis. Finally, simulated results on system identification and channel estimation validate the obtained theoretical results and the merits of the proposed MCSE algorithm.

Calibration Analysis of High-G MEMS Accelerometer Sensor Based on Wavelet and Wavelet Packet Denoising.

High-G accelerometers are mainly used for motion measurement in some special fields, such as projectile penetration and aerospace equipment. This paper mainly explores the wavelet threshold denoising and wavelet packet threshold denoising in wavelet analysis, which is more suitable for high-G piezoresistive accelerometers. In this paper, adaptive decomposition and Shannon entropy criterion are used to find the optimal decomposition layer and optimal tree. Both methods use the Stein unbiased likelihood estimation method for soft threshold denoising. Through numerical simulation and Machete hammer test, the wavelet threshold denoising is more suitable for the dynamic calibration of a high-G accelerometer. The wavelet packet threshold denoising is more suitable for the parameter extraction of the oscillation phase.

Content-based Image Retrieval Considering Colour Difference Histogram of Image Texture and Edge Orientation

Content-based image retrieval is one of the interesting subjects in image processing and machine vision. In image retrieval systems, the query image is compared with images in the database to retrieve images containing similar content. Image comparison is done using features extracted from the query and database images. In this paper, the features are extracted based on the human visual system. Since the human visual system considers the texture and the edge orientation in images for comparison, the colour difference histogram associated with the image’s texture and edge orientation is extracted as a feature. In this paper, the features are selected using the Shannon entropy criterion. The proposed method is tested using the Corel-5K and Corel-10K databases. The precision and recall criteria were used to evaluate the proposed system. The experimental results show the ability of the proposed system for more accurate retrieval rather than recently content-based image retrieval systems.

Optimal Wavelet Parameters for System Identification of Civil Engineering Structures

Damage-induced changes in structure dynamic properties are commonly tracked with time-frequency representations (TFRs). One of the most widely accepted tools for determining a TFR is the continuous wavelet transform (CWT). The success of CWT analysis is highly dependent on selecting the most appropriate values for the parameters that define the mother wavelet. This article presents a detailed analytical and numerical study to select optimal wavelet parameters using the complex Morlet wavelet (CMOR) and the Gabor wavelet. The results obtained suggest that it is possible to define optimal parameter values based on identification target, instantaneous frequency, or average damping ratio. This reduces the computational cost of a reliable CWT analysis when compared with currently employed iterative methodologies based on minimal Shannon entropy criteria.

From fission yield measurements to evaluation: status on statistical methodology for the covariance question

Studies on fission yields have a major impact on the characterization and the understanding of the fission process and are mandatory for reactor applications. Fission yield evaluation represents the synthesis of experimental and theoretical knowledge to perform the best estimation of mass, isotopic and isomeric yields. Today, the output of fission yield evaluation is available as a function of isotopic yields. Without the explicitness of evaluation covariance data, mass yield uncertainties are greater than those of isotopic yields. This is in contradiction with experimental knowledge where the abundance of mass yield measurements is dominant. These last years, different covariance matrices have been suggested but the experimental part of those are neglected. The collaboration between the LPSC Grenoble and the CEA Cadarache starts a new program in the field of the evaluation of fission products in addition to the current experimental program at Institut Laue-Langevin. The goal is to define a new methodology of evaluation based on statistical tests to define the different experimental sets in agreement, giving different solutions for different analysis choices. This study deals with the thermal neutron induced fission of 235U. The mix of data is non-unique and this topic will be discussed using the Shannon entropy criterion in the framework of the statistical methodology proposed.

Rolling Element Bearing Condition Monitoring Based on Vibration Analysis Using Statistical Parameters of Discrete Wavelet Coefficients and Neural Networks

There are several techniques that can be used to determine the condition of a rolling element bearing. In this paper, vibration analysis is used to conduct fault diagnosis of a bearing. Vibration signal noise was eliminated using hard thresholding wavelet analysis. The best mother wavelet for the denoising process was selected using the minimum Shannon entropy criterion. Statistical parameters and other signal properties such as energy and entropy are powerful tools for analyzing vibration signals. These features were calculated in the time and wavelet domains and applied to Artificial Neural Networks (ANNs) as the feature vector to classify the condition of a bearing into one healthy and three faulty conditions. The ANN parameters were separately optimized using three optimization algorithms. The comparison of the results shows that if the ANN parameters are properly optimized, the statistical parameters in the time-frequency domain can optimize accuracy.

Rolling Element Bearing Condition Monitoring Based on Vibration Analysis Using Statistical Parameters of Discrete Wavelet Coefficients and Neural Networks

There are several techniques that can be used to determine the condition of a rolling element bearing. In this paper, vibration analysis is used to conduct fault diagnosis of a bearing. Vibration signal noise was eliminated using hard thresholding wavelet analysis. The best mother wavelet for the denoising process was selected using the minimum Shannon entropy criterion. Statistical parameters and other signal properties such as energy and entropy are powerful tools for analyzing vibration signals. These features were calculated in the time and wavelet domains and applied to Artificial Neural Networks (ANNs) as the feature vector to classify the condition of a bearing into one healthy and three faulty conditions. The ANN parameters were separately optimized using three optimization algorithms. The comparison of the results shows that if the ANN parameters are properly optimized, the statistical parameters in the time-frequency domain can optimize accuracy.

Optimal Wavelet Selection for the Size Estimation of Manufacturing Defects of Tapered Roller Bearings with Vibration Measurement using Shannon Entropy Criteria

Fault diagnosis of bearings is essential in manufacturing to increase quality. Traditionally, fault diagnosis of tapered roller element bearings is performed by signal processing methods, which handle the nonstationary behaviour of the signal. The wavelet transform is an efficient tool for analysing the vibration signal of the bearings because it can detect the sudden changes and transient impulses in the signal caused by faults in the bearing elements. In this article, manufacturing faults on the outer ring of tapered roller bearings due to the grinding process in manufacturing are investigated. Nine different real values wavelets (Symlet-2, Symlet-5, Symlet-8, db02, db06, db10, db14, Meyer, and Morlet) are compared according to the Energy-to-Shannon-Entropy ratio criteria, and which is efficient for detecting the manufacturing faults is determined. Finally, experiments are carried out on a test rig for determining the geometrical size of the manufacturing faults with all wavelets directly from the vibration signature the result of db02, Symlet-5, and Morlet wavelets are presented. When modelling the bearing structure as an under-damped second-order mass-spring-damper mechanical system, its unit impulse response function is compared to the wavelets on the basis of their Energy-to-Shannon-Entropy ratio to determine the fault size from the vibration signal. The proposed technique has been successfully implemented for measuring defect widths. The maximum deviation in result has been found to be 4.12 % for the defect width which was verified with image analysis methods using an optical microscope and contact measurement.

A Fourier Based Wavelet Approach Using Heisenberg’s Uncertainty Principle and Shannon’s Entropy Criterion to Monitor Power System Small Signal Oscillations

This paper presents a novel approach to estimate modal parameters of power systems for monitoring and analyzing the embedded modes of small signal oscillations. The proposed approach applies continuous wavelet transform (CWT) to identify damping and frequency of critical modes based on its time-frequency localization capability. The CWT has modified Morlet function as its mother wavelet. A procedure is also presented to fine-tune settings of the modified Morlet function of the CWT based on Heisenberg's uncertainty principle and Shannon's entropy criterion. Additionally, high computational burden of the time-frequency methods is an important obstacle in online monitoring of power systems by these methods. To remedy this problem, the convolution integral of the CWT is calculated by efficient fast Fourier transform (FFT) routine in the proposed approach leading to a low computational burden. The proposed approach is compared with several other signal processing methods for modal identification of power systems. These comparisons illustrate effectiveness of the proposed approach, regarding run time, persistency against noise and estimation accuracy for online monitoring of small signal oscillations.

OPTIMIZATION OF INFORMATION PREPROCESSING IN CLUSTERING SYSTEMS OF HIGH DIMENSION DATA

The methodic of choice of optimal normalization method for object cluster structure of creation, with high dimension of feature space, is shown. The Shannon entropy criterion and entropy relative change were used as main criterions of estimating the data preprocessing quality during the data transformation. Decreasing of feature space dimension of tested objects was realized by component analysis. Model of system clustering with the use of fuzzy C-means algorithm was constructed, which the help of whith the estimate of clustering quality was established by the use of different data preprocessing methods. It’s shown that the best normalization method for tested data is decimal-scaling method, by which the entropy of processed signal gets minimal significance, and relative change of entropy doesn’t exceed permissible norms during the process of data transformation by component analysis.

A fault location method using Lamb waves and discrete wavelet transform

Non-destructive evaluation methods and signal process techniques are important steps in structural health monitoring systems to assess the structure integrity. This paper presents a method for fault location in aluminum beams based on time of flight of Lamb waves. The dynamic response signal captured from the structure was processed using the discrete wavelet transform. The information accuracy obtained from the processed signal depends on the correct choice of the mother wavelet. The best mother wavelet was selected using the Shannon's entropy criterion. Numerical results for a damage localized in different positions are presented using the spectral finite element method, and an experimental setup was used to assess the accuracy of the method. The results showed that the combination of the non-destructive evaluation technique based on Lamb waves with the discrete wavelet transform is effective in detecting and locating faults in aluminum beams whose results had errors less than 1%.

Rolling element bearing fault diagnosis using wavelet transform

This paper is focused on fault diagnosis of ball bearings having localized defects (spalls) on the various bearing components using wavelet-based feature extraction. The statistical features required for the training and testing of artificial intelligence techniques are calculated by the implementation of a wavelet based methodology developed using Minimum Shannon Entropy Criterion. Seven different base wavelets are considered for the study and Complex Morlet wavelet is selected based on minimum Shannon Entropy Criterion to extract statistical features from wavelet coefficients of raw vibration signals. In the methodology, firstly a wavelet theory based feature extraction methodology is developed that demonstrates the information of fault from the raw signals and then the potential of various artificial intelligence techniques to predict the type of defect in bearings is investigated. Three artificial intelligence techniques are used for faults classifications, out of which two are supervised machine learning techniques i.e. support vector machine, learning vector quantization and other one is an unsupervised machine learning technique i.e. self-organizing maps. The fault classification results show that the support vector machine identified the fault categories of rolling element bearing more accurately and has a better diagnosis performance as compared to the learning vector quantization and self-organizing maps.

Molecular reconstruction of complex hydrocarbon mixtures: An application of principal component analysis

AbstractThree methods for reconstruction of the detailed molecular composition of complex hydrocarbon mixtures, based on their global properties, are compared: a method based on the Shannon entropy criterion, an artificial neural network and a multiple linear regression model. In spite of the broad range of naphthas included in the training set, the application range of the last two methods proved to be limited. Principal component analysis allowed to identify their three‐dimensional ellipsoidal application range. In this subspace, the artificial neural network is more accurate than the multiple linear regression model and the Shannon entropy method. However, outside its application range, the performance of the neural network, as well as the regression model, decreases drastically. In contrast, the performance of the Shannon entropy method is not influenced by the characteristics of the considered naphtha, but rather depends on the number of available commercial indices. The Shannon entropy method yields comparable results to the artificial neural network, provided that a sufficient amount of distillation data is available to supply information on the carbon number distribution. Combining the reconstruction methods with a fundamental simulation model illustrates the necessity of having accurate feedstock reconstruction methods since they allow to capture the full power of fundamental simulation models for the simulation of industrial processes. © 2010 American Institute of Chemical Engineers AIChE J, 2010

Image thresholding based on θ-division of 2-D histogram and maximum Shannon entropy

In view of the obvious wrong segmentation in commonly used region division of 2-D histogram and the non- universality of oblique segmentation method for image thresholding proposed recently, in this paper a much more widely suitable thresholding method is proposed based on the θ-division of 2-D histogram and the maximum Shannon entropy criterion. Firstly, the θ-division method of 2-D histogram is given. The region is divided by four parallel oblique lines and a line, where the angle between its normal line and gray level axis is θ degrees. Image thresholding is performed according to pixel's weighted average value of gray level and neighbour average gray level. The oblique segmentation method can be regarded as a special case of the proposed method at θ=45°. Then the formulae and its fast recursive algorithm of the method are deduced. Finally the segmented results and the running time at different values of θ are listed, which show that the segmented images achieve more accurate borders at smaller values of θ and the anti-noise is better at larger values of θ. The value of θ can be selected according to the real image characteristics and the requirements of segmented results. Compared with the algorithm of conventional 2-D maximum Shannon entropy method, the proposed method not only achieves more accurate segmentation results and more robust anti-noise, but also reduces the running time and memory space significantly.

Phase-space structures - I. A comparison of 6D density estimators

In the framework of particle-based Vlasov systems, this paper reviews and analyses different methods recently proposed in the literature to identify neighbours in 6D space and estimate the corresponding phase-space density. Specifically, it compares smoothed particle hydrodynamics (SPH) methods based on tree partitioning to 6D Delaunay tessellation. This comparison is carried out on statistical and dynamical realizations of single halo profiles, paying particular attention to the unknown scaling, SG, used to relate the spatial dimensions to the velocity dimensions. It is found that, in practice, the methods with local adaptive metric provide the best phase-space estimators. They make use of a Shannon entropy criterion combined with a binary tree partitioning and with subsequent SPH interpolation using 10–40 nearest neighbours. We note that the local scaling SG implemented by such methods, which enforces local isotropy of the distribution function, can vary by about one order of magnitude in different regions within the system. It presents a bimodal distribution, in which one component is dominated by the main part of the halo and the other one is dominated by the substructures of the halo. While potentially better than SPH techniques, since it yields an optimal estimate of the local softening volume (and therefore the local number of neighbours required to perform the interpolation), the Delaunay tessellation in fact generally poorly estimates the phase-space distribution function. Indeed, it requires, prior to its implementation, the choice of a global scaling SG. We propose two simple but efficient methods to estimate SG that yield a good global compromise. However, the Delaunay interpolation still remains quite sensitive to local anisotropies in the distribution. To emphasize the advantages of 6D analysis versus traditional 3D analysis, we also compare realistic 6D phase-space density estimation with the proxy proposed earlier in the literature, Q=ρ/σ3, where ρ is the local 3D (projected) density and 3σ2 is the local 3D velocity dispersion. We show that Q only corresponds to a rough approximation of the true phase-space density, and is not able to capture all the details of the distribution in phase space, ignoring, in particular, filamentation and tidal streams.

Molecular reconstruction of naphtha steam cracking feedstocks based on commercial indices

A new method for feedstock reconstruction of industrially important hydrocarbon fractions is developed using as input the analytically determined commercial indices such as the average molecular weight, the specific density, the H/ C-ratio, the paraffin, isoparaffin, olefins, naphthenes, aromatics (PIONA) weight fractions and a set of ASTM boiling points. The method is based on Shannon's entropy criterion and allows generating a molecular composition of a naphtha fraction that meets all the boundary conditions set by the industrially available commercial indices. The most important components present in a naphtha fraction are selected as possible feedstock components. This approach is very fast because the optimization function can be transformed from a nonlinear equation in the N mole fractions ( N ∼ 10 2) into a nonlinear equation in maximally 14 parameters, making it very attractive for the petrochemical industry. A good correspondence exists between the simulated weight fractions and the actual analytically determined ones provided that the specified commercial indices are sufficiently accurate. The feedstock module is implemented in a simulation package for steam cracking but can be used as a stand-alone tool too. The combination of these two simulation tools makes it possible to obtain a good agreement between the simulated and experimentally determined product yields for a set of 50 pilot plant experiments with different naphtha feedstocks under widely varying operating conditions.

Why Do We Guess Better in Negative Feedback Situations?

We here introduce a beauty contest game with negative feedback and interior equilibrium in a multi-period experiment. This game is isomorphic to classical BCG but fit economic situations such as crop production or professional investment better. The game is still being analysed from the eductive point of view and with respect to the attempt to establish a typology of players according to their depths of reasoning. Our main contribution to the understanding of this game is the formalization of the process by which the information is processed. Using the Shannon entropy criterion, we evaluated information and made a link between the Sperber analysis of reflective and intuitive beliefs and numerical psychological research (Dehaene, 1993). Information that players take into account in their choices is denoted useful information. As this depends on the exploitation of the strategy interval, it will be higher in BCG- than in BCG+ in the first iterations, because strategies are numbers that are naturally scanned several times. As argued by Sperber (1997), there is a point in the reasoning process starting from which reflective beliefs become intuitive. In order to determine the exact location of the specific point from which players in the BCG- can jump to the REE, we assume that sophisticated reasoning is costly. Therefore, an agent stops calculating at step k which is obtained by the intersection between his marginal cost function and his marginal benefit (information) function. However, there are individuals who are not able to reach that point, because their cognitive constraint is saturated beforehand. There are also individuals for whom the cognitive constraint is saturated for a value higher that k, but who stop at step k because, given the structure of the population, they can win the game at a smaller cost. Therefore, a guess in this game corresponds to the solution of the system comprising these two constraints. For our experiments, we found a depth of reasoning smaller than 3, which can, however, be optimal. Results show that the k-step thinking with k

Entropy-based Optimal Group-testing Procedures

Two procedures for the group-testing problem based on the Shannon-entropy criteria are proposed. The model considered is that the N units are realizations of N Bernoulli independent and identically distributed (i.i.d.) chance variables with common, known probability q of an arbitrary unit being good and p =1 – q of it being defective. Both the algorithms introduced have low design complexity and yet provide near-optimal result. For N ≤ 5, one of the procedures introduced is optimal for selected values of q.