Minimum Cross Entropy Research Articles

Medical image segmentation using exchange market algorithm

Color medical images hold noteworthy impact in clinical conclusion and treatment. Medical image segmentation reduces the uncertainty by providing detailed information about the shape, size, or location characteristics. However, traditional methods suffer from low accuracy, high complexity, and are less robust. To overcome these drawbacks, this paper proposes an efficient metaheuristic algorithm, exchange market algorithm (EMA) for multilevel thresholding (MLT) of distinct medical images. Optimal threshold is effectively obtained through the most promising objective functions such as Kapur, Otsu and minimum cross entropy (MCE) aided with EMA. The EMA involves exchange of shares among the investors in stable and unstable market situations to achieve profit. Exploration and exploitation are achieved by second and third groups of stable and unstable modes of EMA. Moreover, the execution time is reduced by the highly competent shareholders retaining their top rank without any changes in their shares. The efficacy of the proposed paper is evaluated on three distinct medical images at 4, 5, 6 and 7th threshold levels and compared with the recent algorithms such as Krill herd (KHA), Teaching-learning based optimization (TLBO) and Cuckoo search algorithm (CSA). Quantitative and qualitative validation by metrics such as computational time, Peak signal to noise ratio (PSNR), Structural similarity index (SSIM) and Wilcoxon rank sum test affirm that the EMA is superior to other algorithms. On the other hand, Otsu based EMA method is found to be more accurate and robust for improved clinical decision making and diagnosis.

An efficient method to minimize cross-entropy for selecting multi-level threshold values using an improved human mental search algorithm

The minimum cross-entropy (MCIT) is introduced as a multi-level image thresholding approach, but it suffers from time complexity, in particular, when the number of thresholds is high. To address this issue, this paper proposes a novel MCIT-based image thresholding based on improved human mental search (HMS) algorithm, a recently proposed population-based metaheuristic algorithm to tackle complex optimisation problems. To further enhance the efficacy, we improve HMS algorithm, IHMSMLIT, with four improvements, including, adaptively selection of the number of mental searches instead of randomly selection, proposing one-step k-means clustering for region clustering, updating based on global and personal experiences, and proposing a random clustering strategy. To assess our proposed algorithm, we conduct an extensive set of experiments with several state-of-the-art and the most recent approaches on a benchmark set of images and in terms of several criteria including objective function, peak signal to noise ratio (PSNR), feature similarity index (FSIM), structural similarity index (SSIM), and stability analysis. The obtained results apparently demonstrate the competitive performance of our proposed algorithm.

An enhanced cloud segmentation algorithm for accurate irradiance forecasting

This paper presents a novel approach to calculate cloud cover under any illumination conditions for short-term irradiance forecasting. A sky-imaging system, in parallel with an irradiance measurement, is set up to collect a database of sky images and irradiance readings. The colour space operations and various image segmentation methods are investigated to improve the visual contrast of the cloud component. Experimental results shows the effectiveness of applying the Normalized Blue-to-Red Ratio (NBRR) colour operation for high-illumination condition and the Red/Green channel for the low-illumination condition. The pre-processed sky images are then fed into Minimum Cross Entropy (MCE) adaptive thresholding segmentation, which effectively differentiates the sky and cloud components for all-sky images. The cloud fraction calculation is employed for the segmented images as an indication of cloud coverage. The proposed method demonstrates a positive linear correlation between cloud fraction and real-time irradiance data.

Krill Herd Algorithm for Color Image Segmentation with Kapur, Otsu and Minimum Cross Entropy Fitness Functions

Color image segmentation is essential to analyze information from the desired image with RGB color space. Generally, visual information can be easily retrieved through the simple, effective technique called thresholding. Segmentation of complex images is accurately achieved, through MultiLevel Thresholding (MLT) with most optimistic objective functions such as Kapur, Otsu and Minimum Cross Entropy (MCE), than Bilevel Thresholding (BLT). However, the complexity in exploring the optimal threshold increases with the increase in levels of threshold. The key to breach this barrier is by the most computationally effective, flexible Krill Herd Algorithm (KHA). The behavior of the krill movement, the foraging activity and the diffusion methods are utilized for global and local searches. KHA incorporates the swarm intelligence and with crossover, mutation operators improve the convergence rate. The performance of the KHA is compared with Teaching-Learning Based Optimization (TLBO) and cuckoo search algorithm (CSA). Experimental results disclose that the Otsu based MLT outperforms the Kapur and the MCE fitness functions. Quantitative and qualitative validations by metrics such as computational time, Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) confirm that Kapur, Otsu and MCE based KHA outperform the existing techniques for real life applications.

Machine Learning-Inspired Hybrid Precoding for mmWave MU-MIMO Systems with Domestic Switch Network.

Hybrid precoding is an attractive technique in MU-MIMO systems with significantly reduced hardware costs. However, it still requires a complex analog network to connect the RF chains and antennas. In this paper, we develop a novel hybrid precoding structure for the downlink transmission with a compact RF structure. Specifically, the proposed structure relies on domestic connections instead of global connections to link RF chains and antennas. Fixed-degree phase shifters provide candidate signals, and simple on-off switches are used to route the signal to antennas, thus RF adders are no longer required. Baseband zero forcing and block diagonalization are used to cancel interference for single-antenna and multiple-antenna users, respectively. We formulate how to design the RF precoder by optimizing the probability distribution through cross-entropy minimization which originated in machine learning. To optimize the energy efficiency, we use the fractional programming technique and exploit the Dinkelbach method-based framework to optimize the number of active antennas. Simulation results show that proposed algorithms can yield significant advantages under different configurations.

Image segmentation with Kapur, Otsu and minimum cross entropy based multilevel thresholding aided with cuckoo search algorithm

Color image segmentation is the primary factor to provide the intended information from the input image. The straightforward method called multilevel thresholding (MLT) is used to analyse the various classes of complex images. But, when the level of threshold increases, computational difficulty increases. Hence, MLT with most promising objective functions such as Kapur, Otsu and minimum cross entropy aided with cuckoo search algorithm (CSA) is used. The efficient metaheuristic cuckoo search algorithm’s controlling parameter balances the local and global search. In this paper, the efficacy of CSA at 4,5,6 and 7 threshold levels with various fitness functions are utilized for precise image segmentation. It is seen from experimental results, the Otsu based cuckoo search algorithm outperform than Kapur and MCE. Quality metrices such as computational time, PSNR (peak signal to noise ratio) and SSIM (structural similarity index) authenticate the exploration and exploitation capability of CSA algorithm for real-world applications.

Intelligent handwritten recognition using hybrid CNN architectures based-SVM classifier with dropout

Text recognition in Arabic handwritten scripts is an active research field. These recognition systems face numerous challenges, including enormous open data-bases, infinite variation in people’s handwriting, and freestyle. In this manuscript, Authors model deep learning architecture which can efficiently be utilized to recognizing Arabic handwritten scripts. This work explored a new model for both single font and multi-font type which concentrate on two common classifiers which are: Support Vector Machine (SVM) along with Convolutional Neural Network (CNN). Furthermore, authors protected the proposed model against the issue of over-fitting because of the strong performance of dropout technique. Both classification and feature extraction are done automatically. In the light of the error backpropagation method analysis, authors also have been proposed an innovative depth neural network training rule for maximum interval minimum classification error. In the meantime, max-margin minimum classification error (M3CE) and cross entropy are analyzed and hybridized to obtain better outcomes. Authors tested the proposed model on AHDB, AHCD, HACDB, and IFN/ENIT databases. The proposed model performance is compared with the accuracies of text recognition gained from state-of-the-art Arabic text recognition. The proposed model delivers favorable results.

Color image segmentation using Kapur, Otsu and Minimum Cross Entropy functions based on Exchange Market Algorithm

Color image segmentation is the primary step to elicit detailed information of the image with RGB color space. The key to acquire the region of interest is the straightforward, instinctual technique called thresholding. Bi-level thresholding can be assessed concretely for simple images, whereas revealing various classes from a complex image is achieved only through multiple threshold values. Hence, multilevel thresholding with the most propitious maximizing objective functions such as Kapur, Otsu and Minimum Cross Entropy (MCE) to extract the precise threshold are employed in this paper. As the segmentation level increases, execution of non-parametric objective functions leads to exponential increase in computational time. Several researches to enhance the speed of objective functions along with various metaheuristics such as Krill Herd Algorithm (KHA), Teaching-Learning Based Optimization (TLBO) and Cuckoo Search Algorithm (CSA) are performed. The need to unlock the computational complexity drifted our view towards the most powerful, robust, recently introduced metaheuristic Exchange Market Algorithm (EMA). EMA involves exchange of shares among investors to gain profit. The strategic approach of share members in stable and unstable modes of the market results in precise exploration and exploitation. Furthermore, low fitness shareholders gaining experience from their high and moderate counterparts, stands out as an exceptional step to reduce the time consumption. This concept is utilized in our paper for the first time to obtain the exact details from the image without any delay. Empirical outcome of the results indicates that exceptional image segmentation is achieved by EMA in less time compared to extensive search techniques such as KHA, TLBO and CSA. Quantitative and qualitative validation furnished by metrics such as Structural Similarity Index (SSIM), computational time, Peak Signal to Noise Ratio (PSNR) and statistical Wilcoxon test affirm that Otsu, Kapur and MCE based EMA outperforms the existing techniques to analyze the real-world images.

A minimum cross-entropy multi-thresholds segmentation algorithm based on improved WOA

Minimum cross-entropy is widely used in image segmentation for its effectiveness. However, when the algorithm is applied to multi-threshold segmentation, there are some problems such as large amount of calculation, time-consuming and poor practicability due to exhaustive search for the optimal threshold. Therefore, in this paper, a hybrid whale optimization algorithm (IWOA) which incorporates whale optimization algorithm (WOA) and invasive weed optimization (IWO) is proposed and the minimum cross-entropy is used as the fitness function of optimization algorithm to select the optimal threshold. It is established that IWOA algorithm is able to select the optimal threshold in more accuracy and segment high quality image than other algorithms.

FMixCutMatch for semi-supervised deep learning

Mixed sample augmentation (MSA) has witnessed great success in the research area of semi-supervised learning (SSL) and is performed by mixing two training samples as an augmentation strategy to effectively smooth the training space. Following the insights on the efficacy of cut-mix in particular, we propose FMixCut, an MSA that combines Fourier space-based data mixing (FMix) and the proposed Fourier space-based data cutting (FCut) for labeled and unlabeled data augmentation. Specifically, for the SSL task, our approach first generates soft pseudo-labels using the model’s previous predictions. The model is then trained to penalize the outputs of the FMix-generated samples so that they are consistent with their mixed soft pseudo-labels. In addition, we propose to use FCut, a new Cutout-based data augmentation strategy that adopts the two masked sample pairs from FMix for weighted cross-entropy minimization. Furthermore, by implementing two regularization techniques, namely, batch label distribution entropy maximization and sample confidence entropy minimization, we further boost the training efficiency. Finally, we introduce a dynamic labeled–unlabeled data mixing (DDM) strategy to further accelerate the convergence of the model. Combining the above process, we finally call our SSL approach as ”FMixCutMatch”, in short FMCmatch. As a result, the proposed FMCmatch achieves state-of-the-art performance on CIFAR-10/100, SVHN and Mini-Imagenet across a variety of SSL conditions with the CNN-13, WRN-28-2 and ResNet-18 networks. In particular, our method achieves a 4.54% test error on CIFAR-10 with 4K labels under the CNN-13 and a 41.25% Top-1 test error on Mini-Imagenet with 10K labels under the ResNet-18. Our codes for reproducing these results are publicly available at https://github.com/biuyq/FMixCutMatch.

A New Method to Improve Automated Classification of Heart Sound Signals: Filter Bank Learning in Convolutional Neural Networks

Introduction: Recent studies have acknowledged the potential of convolutional neural networks (CNNs) in distinguishing healthy and morbid samples by using heart sound analyses. Unfortunately the performance of CNNs is highly dependent on the filtering procedure which is applied to signal in their convolutional layer. The present study aimed to address this problem by applying filter bank learning concept in CNNs. Material and Methods: In proposed method, the filter bank of CNN is updated based on a cross-entropy minimization rule to extract higher-level features from spectral characteristics of the heart sound signal. The deeper level of the extracted features in parallel with their spectral-based nature leads to better discrimination between healthy and morbid heart sounds. The proposed method was applied to three different heart sound datasets of PASCAL-A, PASCAL-B, and Kaggle, including normal and abnormal categories. Results: The proposed method obtained a true positive rate (TPR) between minimally 86% and maximally 96% (if FPR=0%) among all the examined datasets. In addition, the false-positive rate (FPR) was obtained as 7-8% (if TPR=100%) among the mentioned datasets. Finally, the accuracy was achieved in the range of 93-98% when the FPR was 0% and within the range of 96-96.5% when the TRP was 100%. Conclusion: Increased TPR in the proposed method (96% for the proposed method vs. 87% for CNN) in parallel with a decrease in its FPR (7% for the proposed method vs. 10% for CNN) showed the proposed method's superiority against its well-known alternative in automated self-assessment of the heart.

Trading strategies for image segmentation using multilevel thresholding aided with minimum cross entropy

Multilevel thresholding (MLT) is one of the most widely used methods in image segmentation. However, the exhaustive search method is computationally time consuming for selecting the optimal thresholds. Consequently, heuristic algorithms are extensively used to reduce the complexity of the MLT problem. In this paper, an efficient Exchange Market Algorithm (EMA) is proposed to segment images using minimum cross entropy thresholding method. In the EMA, a market risk variable is used to balance the exploration and exploitation capabilities of the algorithm. Moreover, the local search capability is strengthened by the search and absorbent operators of EMA. Meanwhile, the most competent shareholders of EMA retain their best rank without undergoing any changes in their shares. These help in reducing the computational time. The proposed EMA based MLT is tested on benchmark and brain images with different threshold levels. Additionally, EMA approach is compared with other well-known algorithms such as, genetic algorithm, particle swarm optimization, bacterial foraging algorithm, firefly algorithm, honey bee mating optimization and teaching–learning based optimization. The experimental results show that the proposed EMA approach provides better outcomes than other algorithms.

Learning reaction coordinates via cross-entropy minimization: Application to alanine dipeptide

We propose a cross-entropy minimization method for finding the reaction coordinate from a large number of collective variables in complex molecular systems. This method is an extension of the likelihood maximization approach describing the committor function with a sigmoid. By design, the reaction coordinate as a function of various collective variables is optimized such that the distribution of the committor pB * values generated from molecular dynamics simulations can be described in a sigmoidal manner. We also introduce the L2-norm regularization used in the machine learning field to prevent overfitting when the number of considered collective variables is large. The current method is applied to study the isomerization of alanine dipeptide in vacuum, where 45 dihedral angles are used as candidate variables. The regularization parameter is determined by cross-validation using training and test datasets. It is demonstrated that the optimal reaction coordinate involves important dihedral angles, which are consistent with the previously reported results. Furthermore, the points with pB *∼0.5 clearly indicate a separatrix distinguishing reactant and product states on the potential of mean force using the extracted dihedral angles.

Multilevel minimum cross entropy thresholding: A comparative study

Multilevel thresholding method is one of the most popular techniques in image segmentation. However, the multilevel thresholding method is time-consuming, its time complexity increases exponentially with the number of threshold levels. In this paper, in order to improve the computation efficiency of the multilevel minimum cross entropy thresholding, the iterative formula of the multilevel cross entropy thresholding algorithm is proposed and compared with the modern meta-heuristic optimization algorithms. The iterative multilevel cross entropy thresholding algorithm can find the thresholds close to the global optimums with less time. The computation cost of the iterative multilevel cross entropy thresholding algorithm is linear in the number of the threshold levels. We prove the convergence of the iterative algorithm and compare the iterative multilevel cross entropy thresholding algorithm with multilevel cross entropy thresholding methods combined with the state-of-the-art meta-heuristic optimization techniques including particle swarm optimization (PSO), cuckoo search algorithm (CS), differential evolution (DE), crow search algorithm (CSA) and genetic algorithm (GA). Experimental results show that the iterative multilevel cross entropy thresholding algorithm is efficient and effective. Therefore, iterative algorithm for the multilevel cross entropy thresholding is an effective method to improve the computation efficiency.

Genes, the brain, and artificial intelligence in evolution.

Three important systems, genes, the brain, and artificial intelligence (especially deep learning) have similar goals, namely, the maximization of likelihood or minimization of cross-entropy. Animal brains have evolved through predator-prey interactions in which maximizing survival probability and transmission of genes to offspring were the main objectives. Coordinate transformation for a rigid body necessary to win predator-prey battles requires a huge amount of matrix operations in the brain similar to those performed by a powerful GPU. Things (molecules), information (genes), and energy (ATP) are essential for using Maxwell's demon model to understand how a living system maintains a low level of entropy. However, while the history of medicine and biology saw molecular biology and genetics disciplines flourish, the study of energy has been limited, despite estimates that >10% all human genes code energy-related proteins. Since there are a large number of molecular and genetic diseases, many energy-related diseases must exist as well. In addition to mitochondrial disease, common diseases such as neurodegenerative diseases, muscle diseases, cardiomyopathy, and diabetes are candidates for diseases related to cellular energy shortage. We are developing ATP enhancer, a drug to treat such diseases. I predict that in the future, the frontier of medicine and biology will involve energy and entropy, and the frontier of science will be about the cognitive processes that scientists' brains use to study mathematics and physics. That will be understood by comparing the abilities that were necessary to survive battles between predators and prey during evolutionary history.

Machine Segmentation of Pelvic Anatomy in MRI-Assisted Radiosurgery (MARS) for Prostate Cancer Brachytherapy

To investigate machine segmentation of pelvic anatomy in magnetic resonance imaging (MRI)-assisted radiosurgery (MARS) for prostate cancer using prostate brachytherapy MRIs acquired with different pulse sequences and image contrasts. Two hundred 3-dimensional (3D) preimplant and postimplant prostate brachytherapy MRI scans were acquired with a T2-weighted sequence, a T2/T1-weighted sequence, or a T1-weighted sequence. One hundred twenty deep machine learning models were trained to segment the prostate, seminal vesicles, external urinary sphincter, rectum, and bladder using the MRI scans acquired with T2-weighted and T2/T1-weighted image contrast. The deep machine learning models consisted of 18 fully convolutional networks (FCNs) with different convolutional encoders. Both 2-dimensional and 3D U-Net FCNs were constructed for comparison. Six objective functions were investigated: cross-entropy, Jaccard distance, focal loss, and 3 variations of Tversky distance. The performance of the models was compared using similarity metrics, including pixel accuracy, Jaccard index, Dice similarity coefficient (DSC), 95% Hausdorff distance, relative volume difference, Matthews correlation coefficient, precision, recall, and average symmetrical surface distance. We selected the highest-performing architecture and investigated how the amount of training data, use of skip connections, and data augmentation affected segmentation performance. In addition, we investigated whether segmentation on T1-weighted MRI was possible with FCNs trained on only T2-weighted and T2/T1-weighted image contrast. Overall, an FCN with a DenseNet201 encoder trained via cross-entropy minimization yielded the highest combined segmentation performance. For the 53 3D test MRI scans acquired with T2-weighted or T2/T1-weighted image contrast, the DSCs of the prostate, external urinary sphincter, seminal vesicles, rectum, and bladder were 0.90 ± 0.04, 0.70 ± 0.15, 0.80 ± 0.12, 0.91 ± 0.06, and 0.96 ± 0.04, respectively, after model fine-tuning. For the 5 T1-weighted images, the DSCs of these organs were 0.82 ± 0.07, 0.17 ± 0.15, 0.46 ± 0.21, 0.87 ± 0.06, and 0.88 ± 0.05, respectively. Machine segmentation of the prostate and surrounding anatomy on 3D MRIs acquired with different pulse sequences for MARS low-dose-rate prostate brachytherapy is possible with a single FCN.

Evaluation of black-start scheme based on risk utility function

Black-start scheme plays an important role for the rapid restoration of the power system after a large-area blackout. The subjective role of decision makers is easily ignored by traditional calculation method of indicator weight and black-start scheme evaluation model. Therefore, this paper proposes a new calculation method of black-start indicator weight and a novel black-start scheme evaluation model based on risk utility function. This paper uses different risk utility functions to consider the subjective will of decision makers, and employs the combination of Gini coefficient and Pearson coefficient to calculate indicator weight based on the difference of weight values and the conflict among weights. Furthermore, the comprehensive indicator weight is calculated with considering of indicator weights integrity and weight itself based on minimum cross entropy criterion and game theory respectively. At the same time, this paper proposes the weight adjustment coefficient to combine the two different comprehensive weights, and studies the influence of the change of the weight adjustment coefficient on the evaluation result. The traditional VIKOR evaluation method is improved by adopting the prospect theory to solve the problem that the influence of decision-makers' experience and knowledge on evaluation results is easily neglected. The influence of the change of the decision coefficient on the ranking result of the black start scheme is discussed based on the optimal weight adjustment coefficient determined by combining the relative entropy and the gray correlation. Finally, Example results verify the rationality and effectiveness of the proposed calculation method and evaluation model in this paper.

An information theory perspective on the balanced minimum evolution problem

We show that the Balanced Minimum Evolution Problem (BMEP) is a cross-entropy minimization problem. This new perspective both extends the previous interpretations of the BMEP length function described in the literature and enables the identification of an efficiently computable family of lower bounds on the value of the optimal solution to the problem.

An efficient Harris hawks-inspired image segmentation method

Segmentation is a crucial phase in image processing because it simplifies the representation of an image and facilitates its analysis. The multilevel thresholding method is more efficient for segmenting digital mammograms compared to the classic bi-level thresholding since it uses a higher number of intensities to represent different regions in the image. In the literature, there are different techniques for multilevel segmentation; however, most of these approaches do not obtain good segmented images. In addition, they are computationally expensive. Recently, statistical criteria such as Otsu, Kapur, and cross-entropy have been utilized in combination with evolutionary and swarm-based strategies to investigate the optimal threshold values for multilevel segmentation. In this paper, an efficient methodology for multilevel segmentation is proposed using the Harris Hawks Optimization (HHO) algorithm and the minimum cross-entropy as a fitness function. To substantiate the results and effectiveness of the HHO-based method, it has been tested over a benchmark set of reference images, with the Berkeley segmentation database, and with medical images of digital mammography. The proposed HHO-based solver is verified based on other comparable optimizers and two machine learning algorithms K-means and the Fuzzy IterAg. The comparisons were performed based on three groups. This first one is to provide evidence of the optimization capabilities of the HHO using the Wilcoxon test, and the second is to verify segmented image quality using the PSNR, SSIM, and FSIM metrics. Then, the third way is to verify the segmented image comparing it with the ground-truth through the metrics PRI, GCE, and VoI. The experimental results, which are validated by statistical analysis, show that the introduced method produces efficient and reliable results in terms of quality, consistency, and accuracy in comparison with the other methods. This HHO-based method presents an improvement over other segmentation approaches that are currently used in the literature.

Real-Time Localization Approach for Maize Cores at Seedling Stage Based on Machine Vision

To realize quick localization of plant maize, a new real-time localization approach is proposed for maize cores at the seedling stage, which can meet the basic demands for localization and quantitative fertilization in precision agriculture and reduce environmental pollution and the use of chemical fertilizers. In the first stage, by taking pictures of maize at the seedling stage in a field with a monocular camera, the maize is segmented from the weed background of the picture. And then the three most-effective methods (i.e., minimum cross entropy, ISODATA, and the Otsu algorithm) are found from six common segmentation algorithms after comparing the accuracy rate of extracting maize and the time efficiency of segmentation. In the second stage, plant core from segmented maize image is recognized, and localized, based on different brightness of the rest part of maize core and plant. Then the geometric center of maize core is considered as localization point. the best effect of extracting maize core was found from the minimum cross entropy method based on gray level. According to experimental validation using many field pictures, under weedy conditions on sunny days, the proposed method has a minimum recognition rate of 88.37% for maize cores and is more robust at excluding weeds.