Minimum Cross Entropy Research Articles

Empowered chaotic local search-based differential evolution algorithm with entropy-based hybrid objective function for brain tumor segmentation

In neuro-oncology, the precise segmentation of brain tumors from Magnetic Resonance Images is crucial for diagnosis, treatment planning, and monitoring disease progression. Accurate segmentation helps determine the tumor’s size, location, and growth potential, which is essential for formulating effective treatment strategies. In response to this challenge, we developed a novel approach using Chaotic Local Search-Enhanced Differential Evolution (CJADE). CJADE, particularly its variant CJADE-M, which employs chaotic maps selected through a probability-based approach, has proven effective in optimizing brain tumor segmentation. Our study shows that CJADE-M outperforms traditional metaheuristic algorithms on various evaluation metrics. We further enhanced CJADE-M with an entropy-based hybrid objective function, which improved accuracy and reduced computational time in tumor segmentation compared to conventional methods like Minimum Cross-Entropy and Kapur. This makes our method suitable for real-time medical imaging analysis. Our findings indicate that CJADE-M, equipped with the hybrid objective function, achieves superior segmentation performance for both benign lobulated and malignant irregular tumors across metrics such as PSNR, FSIM, QILV, and HPSI. By providing a more accurate and efficient tool, our approach can significantly enhance the outcomes of brain tumor diagnosis and treatment, improving patient care in neuro-oncology.

A two-stage co-adversarial perturbation to mitigate out-of-distribution generalization of large-scale graph

In the realm of graph out-of-distribution (OOD), despite recent strides in advancing graph neural networks (GNNs) for the modeling of graph data, training GNNs on large-scale datasets presents a formidable hurdle due to the pervasive challenge of overfitting. To address these issues, researchers have explored adversarial training, a technique that enriches training data with worst-case adversarial examples. However, while prior work on adversarial training primarily focuses on safeguarding GNNs against malicious attacks, its potential to enhance the OOD generalization abilities of GNNs in the context of graph analytics remains less explored. In our research, we delve into the inner workings of GNNs by examining the landscapes of weight and feature losses, which respectively illustrate how the loss function changes concerning model weights and node features. Our investigation reveals a noteworthy phenomenon: GNNs are inclined to become trapped in sharp local minima within these loss landscapes, resulting in suboptimal OOD generalization performance. To address this challenge, we introduce the concept of co-adversarial perturbation optimization, which considers both model weights and node features, and we design an alternating adversarial perturbation algorithm for graph out-of-distribution generalization. This algorithm operates iteratively, smoothing the weight and feature loss landscapes alternately. Moreover, our training process unfolds in two distinct stages. The first stage centers on standard cross-entropy minimization, ensuring rapid convergence of GNN models. In the second stage, we employ our alternating adversarial training strategy to prevent the models from becoming ensnared in locally sharp minima. Our extensive experiments provide compelling evidence that our CAP approach can generally enhance the OOD generalization performance of GNNs across a diverse range of large-scale graphs.

Multi-strategy learning-based particle swarm optimization algorithm for COVID-19 threshold segmentation

With advancements in science and technology, the depth of human research on COVID-19 is increasing, making the investigation of medical images a focal point. Image segmentation, a crucial step preceding image processing, holds significance in the realm of medical image analysis. Traditional threshold image segmentation proves to be less efficient, posing challenges in selecting an appropriate threshold value. In response to these issues, this paper introduces Inner-based multi-strategy particle swarm optimization (IPSOsono) for conducting numerical experiments and enhancing threshold image segmentation in COVID-19 medical images. A novel dynamic oscillatory weight, derived from the PSO variant for single-objective numerical optimization (PSOsono) is incorporated. Simultaneously, the historical optimal positions of individuals in the particle swarm undergo random updates, diminishing the likelihood of algorithm stagnation and local optima. Moreover, an inner selection learning mechanism is proposed in the update of optimal positions, dynamically refining the global optimal solution. In the CEC 2013 benchmark test, PSOsono demonstrates a certain advantage in optimization capability compared to algorithms proposed in recent years, proving the effectiveness and feasibility of PSOsono. In the Minimum Cross Entropy threshold segmentation experiments for COVID-19, PSOsono exhibits a more prominent segmentation capability compared to other algorithms, showing good generalization across 6 CT images and further validating the practicality of the algorithm.

Deep multimodal spatio-temporal Harris Hawk Optimized Pose Recognition framework for self-learning fitness exercises

Human pose recognition from videotapes has become an emerging research topic for tracking human movements. The objective of this work is to develop a deep multimodal Spatio-Temporal Harris Hawk Optimized Pose Recognition (STHHO-PR) framework for self-learning fitness exercises. The presented STHHO-PR framework uses audio modality and visual modality to classify the different poses. In audio modality, the VGG-16 network paradigm is used to extract the audio traits for fitness pose recognition. In visual modality, Harris Hawks Optimization (HHO) along with the Minimum Cross Entropy (MCE) method is employed to find out the optimum threshold values for body parts segmentation. These segmented body parts highlight the human joint points that are connected through the skeletonization process to extract the skeletal information. The extracted spatio-temporal features from audio modality and visual modality are optimally fused and used in the classification process. Weighted Majority Voting Ensemble (WMVE) classifier is adopted to build the classification model. This work is experimented with yoga videos acquired from publicly available datasets. The results show that the presented STHHO-PR framework outperforms other state-of-art procedures in terms of prediction accuracy.

A multi-feature fusion model based on long and short term memory network and improved artificial bee colony algorithm for Esnglish text classification

The traditional methods of English text classification have two disadvantages. One is that they cannot fully represent the semantic information of the text. The other is that they cannot fully extract and integrate the global and local information of the text. Therefore, we propose a multi-feature fusion model based on long and short term memory network and improved artificial bee colony algorithm for English text classification. In this method, the character-level vector and word-level vector representations of English text are calculated using a pre-training model to obtain a more comprehensive text feature vector representation. Then the multi-head attention mechanism is used to capture the dependencies in the text sequence to improve the semantic understanding of the text. Through feature fusion, the channel features are optimized and the spatial features and time series features are combined to improve the classification performance of the hybrid model. In the stage of network training, the weighted linear combination of maximum Shannon entropy and minimum cross entropy is used as the return degree evaluation function of the bee colony algorithm, and the scale factor is introduced to adjust the solution search strategy of leading bees and following bees, and the improved artificial bee colony algorithm is combined with the classification network to realize the automatic optimization and adjustment of network parameters. Experiments are carried out on public data set. Compared with traditional convolutional neural networks, the classification accuracy of the new model increases by 2% on average, and the accuracy of data set increases by 2.4% at the highest.

An efficient hybrid differential evolution-golden jackal optimization algorithm for multilevel thresholding image segmentation.

Image segmentation is a crucial process in the field of image processing. Multilevel threshold segmentation is an effective image segmentation method, where an image is segmented into different regions based on multilevel thresholds for information analysis. However, the complexity of multilevel thresholding increases dramatically as the number of thresholds increases. To address this challenge, this article proposes a novel hybrid algorithm, termed differential evolution-golden jackal optimizer (DEGJO), for multilevel thresholding image segmentation using the minimum cross-entropy (MCE) as a fitness function. The DE algorithm is combined with the GJO algorithm for iterative updating of position, which enhances the search capacity of the GJO algorithm. The performance of the DEGJO algorithm is assessed on the CEC2021 benchmark function and compared with state-of-the-art optimization algorithms. Additionally, the efficacy of the proposed algorithm is evaluated by performing multilevel segmentation experiments on benchmark images. The experimental results demonstrate that the DEGJO algorithm achieves superior performance in terms of fitness values compared to other metaheuristic algorithms. Moreover, it also yields good results in quantitative performance metrics such as peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and feature similarity index (FSIM) measurements.

Integrating remote sensing temporal trajectory and survey statistics to update land use/land cover maps

ABSTRACTRemote sensing and land resource surveys have been used in recent decades for land use/land cover (LULC) mapping; however, keeping the developed LULC up-to-date and consistent with land survey statistics remains challenging. This study developed a practical and effective framework to automatically update existing LULC products and bridge the gap between remote sensing classification results and land survey data. This study employed Landsat imagery time series, change detection algorithms, sample migration, and random forests to develop a framework for updating existing LULC products in China from 1980–2015 to 1980–2022. The updated LULC maps reflect the post-2015 LULC changes well and maintain continuity with the pre-2015 products. Additionally, a statistical space allocation method based on the minimum cross-entropy strategy was proposed to optimize the LULC maps, increasing the correlation coefficient (r) with China’s second and third national land survey statistics from 0.41–0.89 to 0.86–0.99. Thus, the framework and products developed in this study provide valuable tools for sustainable land use and policy planning.

Gumbel (EVI)-Based Minimum Cross-Entropy Thresholding for the Segmentation of Images with Skewed Histograms

In this study, we delve into the realm of image segmentation, a field characterized by a multitude of approaches; one frequently used technique is thresholding-based image segmentation. This process divides intensity levels into different regions based on a specified threshold value. Minimum Cross-Entropy Thresholding (MCET) stands out as an independent objective function that can be applied with any distribution and is regarded as a mean-based thresholding method. In certain cases, images exhibit diverse structures that result in different histogram distributions. Some images possess symmetric histograms, while others feature asymmetric ones. Traditional mean-based thresholding methods are well-suited for symmetric image histograms, relying on Gaussian distribution definitions for mean estimations. However, in situations involving asymmetric distributions, such as left and right-skewed histograms, a different approach is required. In this paper, we propose the utilization of a Maximum Likelihood Estimation (MLE) of Gumbel’s distribution or Extreme Value Type I (EVI) distribution for the objective function of an MCET. Our goal is to introduce a dedicated image-thresholding model designed to enhance the accuracy and efficiency of image-segmentation tasks. This model determines optimal thresholds for image segmentation, facilitating precise data analysis for specific image types and yielding improved segmentation results by considering the impact of mean values on thresholding objective functions. We compare our proposed model with original methods and related studies in the literature. Our model demonstrates better performance in terms of segmentation accuracy, as assessed through both unsupervised and supervised evaluations for image segmentation.

Multilevel Thresholding of Color Image Segmentation Using Memory-based Grey Wolf Optimizer With Otsu Method, Kapur, and M.Masi Entropy

Determining the optimal threshold value for image segmentation has become more attention in recent years because of its varied uses. Otsu-based thresholding methods, minimum cross entropy, and Kapur entropy are efficient for solving bi-level thresholding image segmentation problems (BL-ISP), but not with multi-level thresholding image segmentation problems (ML-ISP). The main problem is exponentially increasing computational complexity. This study uses the memory-based Gray Wolf Optimizer (mGWO) to determine the optimal threshold value for solving ML-ISP on RGB images. The mGWO method is a variant of the standard grey wolf optimizer (GWO) that utilizes the best track record of each individual grey wolf for the global exploration and local exploitation phases of the problem solution space. The solution candidates are represented by each grey wolf using the image intensity values and optimized according to mGWO characteristics. Three objective functions, namely the Otsu method, Kapur Entropy, and M.Masi Entropy are used to evaluate the solutions generated in the optimization process. The GridSearch method is used to determine the optimal parameter combination of each method based on 10 training images. Evaluation of the performance of the mGWO method was measured using several benchmark images and compared with five standard swarm intelligence (SI) methods as benchmarks. Analysis of the results was carried out qualitatively and quantitatively based on the average PSNR, RMSE, SSIM, UQI, fitness value, and CPU processing time from 30 tests. The results were analyzed further with the Wilcoxon signed-rank test. The experimental results show that the performance of the mGWO method outperforms the benchmark method in most experiments and metrics. The mGWO variant also proved to be superior to the standard GWO in resolving multi-level color image segmentation problems. The mGWO performance results are also compared with other state-of-the-art SI methods in solving ML-ISP on grayscale images and was able to outperform those methods in most experiments.

THE COMPARISON OF THE EFFECTS OF THRESHOLDING METHODS ON SEGMENTATION USING THE MOTH FLAME OPTIMIZATION ALGORITHM

Segmentation is an important preprocessing step that directly affects the success in image processing applications. There are many methods and approaches used for the segmentation process. Thresholding is a frequently used approach among these methods. There are several suggested approaches to thresholding. In this study, six different thresholding approaches were used as the fitness functions using the moth flame algorithm and the results obtained from these approaches were compared. In experimental studies, seven different threshold levels of 10 different images were studied. In comparisons made with three different metrics, it was seen that the Otsu method was generally more successful. It has also been observed that the minimum cross entropy and Renyi entropies can be used as alternatives.

Certified Dimension Reduction for Bayesian Updating with the Cross-Entropy Method

In inverse problems, the parameters of a model are estimated based on observations of the model response. The Bayesian approach is powerful for solving such problems; one formulates a prior distribution for the parameter state that is updated with the observations to compute the posterior parameter distribution. Solving for the posterior distribution can be challenging when, e.g., prior and posterior significantly differ from one another and/or the parameter space is high-dimensional. We use a sequence of importance sampling measures that arise by tempering the likelihood to approach inverse problems exhibiting a significant distance between prior and posterior. Each importance sampling measure is identified by cross-entropy minimization as proposed in the context of Bayesian inverse problems in Engel et al. [J. Comput. Phys., 473 (2023), 111746]. To efficiently address problems with high-dimensional parameter spaces, we set up the minimization procedure in a low-dimensional subspace of the original parameter space. The principal idea is to analyze the spectrum of the second-moment matrix of the gradient of the log-likelihood function to identify a suitable subspace. Following Zahm et al. [Math. Comp., 91 (2022), pp. 1789–1835], an upper bound on the Kullback–Leibler divergence between full-dimensional and subspace posterior is provided, which can be utilized to determine the effective dimension of the inverse problem corresponding to a prescribed approximation error bound. We suggest heuristic criteria for optimally selecting the number of model and model gradient evaluations in each iteration of the importance sampling sequence. We investigate the performance of this approach using examples from engineering mechanics set in various parameter space dimensions.

Real-time single-molecule 3D tracking in E. coli based on cross-entropy minimization

Reaching sub-millisecond 3D tracking of individual molecules in living cells would enable direct measurements of diffusion-limited macromolecular interactions under physiological conditions. Here, we present a 3D tracking principle that approaches the relevant regime. The method is based on the true excitation point spread function and cross-entropy minimization for position localization of moving fluorescent reporters. Tests on beads moving on a stage reaches 67 nm lateral and 109 nm axial precision with a time resolution of 0.84 ms at a photon count rate of 60 kHz; the measurements agree with the theoretical and simulated predictions. Our implementation also features a method for microsecond 3D PSF positioning and an estimator for diffusion analysis of tracking data. Finally, we successfully apply these methods to track the Trigger Factor protein in living bacterial cells. Overall, our results show that while it is possible to reach sub-millisecond live-cell single-molecule tracking, it is still hard to resolve state transitions based on diffusivity at this time scale.

Bayesian updating of reliability by cross entropy-based importance sampling

Bayesian analysis enables a consistent updating of the failure probability of engineering systems when new data is available. To this end, we introduce an adaptive importance sampling (IS) method based on the principle of cross entropy (CE) minimization. The key contribution is a novel IS density associated with the posterior probability density function (PDF) of the uncertain parameters, that facilitates efficient sampling from the important region of the failure domain, especially when the failure event is rare. The IS density is designed via a two-step procedure. The first step involves construction of a sample-based approximation of the posterior, which we build using the CE method. Here the aim is to determine the parameters of a chosen parametric distribution family that minimize its Kullback–Leibler divergence from the posterior PDF. The second step of the proposed method constructs the desired IS density for sampling the failure domain through a second round of CE minimization, starting from the approximate posterior obtained in the first step. An adaptive, multi-level approach is employed to solve the two CE optimization problems. The IS densities deduced in the two steps are then applied to construct an efficient estimator for the posterior probability of failure. Through numerical studies, we investigate and demonstrate the efficacy of the method in accurately estimating the reliability of engineering systems with rare failure events.

Learning curves for the multi-class teacher–student perceptron

One of the most classical results in high-dimensional learning theory provides a closed-form expression for the generalisation error of binary classification with a single-layer teacher–student perceptron on i.i.d. Gaussian inputs. Both Bayes-optimal (BO) estimation and empirical risk minimisation (ERM) were extensively analysed in this setting. At the same time, a considerable part of modern machine learning practice concerns multi-class classification. Yet, an analogous analysis for the multi-class teacher–student perceptron was missing. In this manuscript we fill this gap by deriving and evaluating asymptotic expressions for the BO and ERM generalisation errors in the high-dimensional regime. For Gaussian teacher, we investigate the performance of ERM with both cross-entropy and square losses, and explore the role of ridge regularisation in approaching Bayes-optimality. In particular, we observe that regularised cross-entropy minimisation yields close-to-optimal accuracy. Instead, for Rademacher teacher we show that a first-order phase transition arises in the BO performance.

Integrated Energy System Evaluation Method Based on Minimum Cross-Entropy and Prospect Theory

An integrated energy system can improve the reliability and flexibility of the energy supply, and has good economic and environmental benefits. Integrated energy system evaluation is of great significance to the implementation of an integrated energy system project. The integrated energy system evaluation method is presented based on the minimum cross-entropy model and the prospect theory. Firstly, the minimum cross-entropy model is used to obtain the comprehensive column vector of each evaluation index. Secondly, the index weight is determined according to the different weight methods. Finally, through the establishment of positive and negative prospect value matrices, according to the comprehensive prospect value, the advantages and disadvantages of integrated energy system schemes are sorted. The effectiveness of the proposed method is verified by hotel-integrated energy system data.

On the Image Reconstruction of Capacitively Coupled Electrical Resistance Tomography (CCERT) with Entropy Priors.

Regularization with priors is an effective approach to solve the ill-posed inverse problem of electrical tomography. Entropy priors have been proven to be promising in radiation tomography but have received less attention in the literature of electrical tomography. This work aims to investigate the image reconstruction of capacitively coupled electrical resistance tomography (CCERT) with entropy priors. Four types of entropy priors are introduced, including the image entropy, the projection entropy, the image-projection joint entropy, and the cross-entropy between the measurement projection and the forward projection. Correspondingly, objective functions with the four entropy priors are developed, where the first three are implemented under the maximum entropy strategy and the last one is implemented under the minimum cross-entropy strategy. Linear back-projection is adopted to obtain the initial image. The steepest descent method is utilized to optimize the objective function and obtain the final image. Experimental results show that the four entropy priors are effective in regularization of the ill-posed inverse problem of CCERT to obtain a reasonable solution. Compared with the initial image obtained by linear back projection, all the four entropy priors make sense in improving the image quality. Results also indicate that cross-entropy has the best performance among the four entropy priors in the image reconstruction of CCERT.

Investment benefit evaluation of wind power energy storage based on improved minimum cross entropy method

Investment benefit evaluation of wind power energy storage based on improved minimum cross entropy method

Investment benefit evaluation of wind power energy storage based on improved minimum cross entropy method

Investment benefit evaluation of wind power energy storage based on improved minimum cross entropy method

Multilevel thresholding for crop image segmentation based on recursive minimum cross entropy using a swarm-based technique

The segmentation of crop images is the most sought-after technology in the field of computer vision and pattern recognition. The multilevel thresholding (MLT) of crop images is a tedious job due to the complex background and uneven distribution of intensity levels. The recursive minimum cross entropy (R-MCE) approach is a notable technique for MLT. Therefore, in this paper, R-MCE has been used with an efficient cuckoo search algorithm (CSA), based on Lévy flight for determining the best thresholds. This technique reduces the complexity of computation and increases the accuracy, when it is used for MLT. The dexterity of the proposed technique has been examined over 20 crop images with different illumination and complex backgrounds. Results of the proposed method have been expressed in terms of feature similarity index (FSIM), structural similarity index (SSIM), mean square error (MSE), peak signal-to-noise ratio (PSNR), and CPU time. To investigate the efficiency of proposed technique, the segmented results have been compared with well-known thresholding methods, such as wind-driven optimization (WDO), bacterial foraging optimization (BFO), beta differential evolution (BDE), and artificial bee colony (ABC). Simulation results show that the MLT of the proposed approach can accurately and efficiently segment the complex background crop images.

AUC optimization for deep learning-based voice activity detection

Voice activity detection (VAD) based on deep neural networks (DNN) have demonstrated good performance in adverse acoustic environments. Current DNN-based VAD optimizes a surrogate function, e.g., minimum cross-entropy or minimum squared error, at a given decision threshold. However, VAD usually works on-the-fly with a dynamic decision threshold, and the receiver operating characteristic (ROC) curve is a global evaluation metric for VAD at all possible decision thresholds. In this paper, we propose to maximize the area under the ROC curve (MaxAUC) by DNN, which can maximize the performance of VAD in terms of the entire ROC curve. However, the objective of the AUC maximization is nondifferentiable. To overcome this difficulty, we relax the nondifferentiable loss function to two differentiable approximation functions—sigmoid loss and hinge loss. To study the effectiveness of the proposed MaxAUC-DNN VAD, we take either a standard feedforward neural network or a bidirectional long short-term memory network as the DNN model with either the state-of-the-art multi-resolution cochleagram or short-term Fourier transform as the acoustic feature. We conducted noise-independent training to all comparison methods. Experimental results show that taking AUC as the optimization objective results in higher performance than the common objectives of the minimum squared error and minimum cross-entropy. The experimental conclusion is consistent across different DNN structures, acoustic features, noise scenarios, training sets, and languages.