Entropy-based Algorithm Research Articles

Generating Pattern-Based Conventions for Predictable Planning in Human-Robot Collaboration

For humans to effectively work with robots, they must be able to predict the actions and behaviors of their robot teammates rather than merely react to them. While there are existing techniques enabling robots to adapt to human behavior, there is a demonstrated need for methods that explicitly improve humans’ ability to understand and predict robot behavior at multi-task timescales. In this work, we propose a method leveraging the innate human propensity for pattern recognition in order to improve team dynamics in human-robot teams and to make robots more predictable to the humans that work with them. Patterns are a cognitive tool that humans use and rely on often, and the human brain is in many ways primed for pattern recognition and usage. We propose Pattern-Aware Convention-setting for Teaming (PACT), an entropy-based algorithm that identifies and imposes appropriate patterns over a robot’s planner or policy over long time horizons. These patterns are autonomously generated and chosen via an algorithmic process that considers human-perceptible features and characteristics derived from the tasks to be completed, and as such, produces behavior that is easier for humans to identify and predict. Our evaluation shows that PACT contributes to significant improvements in team dynamics and teammate perceptions of the robot, as compared to robots that utilize traditionally ‘optimal’ plans and robots utilizing unoptimized patterns.

An entropy - based fuzzy QFD model on used product remanufacturing design

Compared with new products, consumers put forward higher requirements on remanufactured products. Aimed at effectively transforming customer requirements (CRs) to technical characteristics (TCs) of remanufactured product design, an entropy-based fuzzy quality function deployment (QFD) model is proposed. In this model, triangular fuzzy numbers are suggested to express the vagueness information of CRs first, and then the importance of CRs is determined by an entropy-based algorithm, and the relationship between CRs and TCs is well explained by fuzzy QFD; finally, the importance of TCs is determined to clarify the designing emphasis. A remanufacturing engine is conducted to illuminate the practicability of this model. The results show that “Restorability” and “Reliability” will be treated as the key TCs to control, with the importance degree of 0.16 and 0.15. The entropy-based fuzzy QFD model can also be applied to similar product designs.

The Evaluation of Entropy-based Algorithm towards the Production of Closed-Loop Edge

This research concerns the common problem of edge detection that produces a disjointed and incomplete edge, leading to the misdetection of visual objects. The entropy-based algorithm can potentially solve this problem by classifying the pixel belonging to which objects in an image. Hence, the paper aims to evaluate the performance of entropy-based algorithm to produce the closed-loop edge representing the formation of object boundary. The research utilizes the concept of Entropy to sense the uncertainty of pixel membership to the existing objects to classify pixels as the edge or object. Six entropy-based algorithms are evaluated, i.e., the optimum Entropy based on Shannon formula, the optimum of relative-entropy based on Kullback-Leibler divergence, the maximum of optimum entropy neighbor, the minimum of optimum relative-entropy neighbor, the thinning of optimum entropy neighbor, and the thinning of optimum relative-entropy neighbor. The experiment is held to compare the developed algorithms against Canny as a benchmark by employing five performance parameters, i.e., the average number of detected objects, the average number of detected edge pixels, the average size of detected objects, the ratio of the number of edge pixel per object, and the average of ten biggest sizes. The experiment shows that the entropy-based algorithms significantly improve the production of closed-loop edges, and the optimum of relative-entropy neighbor based on Kullback-Leibler divergence becomes the most desired approach among others due to the production of more considerable closed-loop edge in the average. This finding suggests that the entropy-based algorithm is the best choice for edge-based segmentation. The effectiveness of Entropy in the segmentation task is addressed for further research.

The Evaluation of Entropy-based Algorithm towards the Production of Closed-Loop Edge

This research concerns the common problem of edge detection that produces a disjointed and incomplete edge, leading to the misdetection of visual objects. The entropy-based algorithm can potentially solve this problem by classifying the pixel belonging to which objects in an image. Hence, the paper aims to evaluate the performance of entropy-based algorithm to produce the closed-loop edge representing the formation of object boundary. The research utilizes the concept of Entropy to sense the uncertainty of pixel membership to the existing objects to classify pixels as the edge or object. Six entropy-based algorithms are evaluated, i.e., the optimum Entropy based on Shannon formula, the optimum of relative-entropy based on Kullback-Leibler divergence, the maximum of optimum entropy neighbor, the minimum of optimum relative-entropy neighbor, the thinning of optimum entropy neighbor, and the thinning of optimum relative-entropy neighbor. The experiment is held to compare the developed algorithms against Canny as a benchmark by employing five performance parameters, i.e., the average number of detected objects, the average number of detected edge pixels, the average size of detected objects, the ratio of the number of edge pixel per object, and the average of ten biggest sizes. The experiment shows that the entropy-based algorithms significantly improve the production of closed-loop edges, and the optimum of relative-entropy neighbor based on Kullback-Leibler divergence becomes the most desired approach among others due to the production of more considerable closed-loop edge in the average. This finding suggests that the entropy-based algorithm is the best choice for edge-based segmentation. The effectiveness of Entropy in the segmentation task is addressed for further research.

The Evaluation of Entropy-based Algorithm towards the Production of Closed-Loop Edge

This research concerns the common problem of edge detection that produces a disjointed and incomplete edge, leading to the misdetection of visual objects. The entropy-based algorithm can potentially solve this problem by classifying the pixel belonging to which objects in an image. Hence, the paper aims to evaluate the performance of entropy-based algorithm to produce the closed-loop edge representing the formation of object boundary. The research utilizes the concept of Entropy to sense the uncertainty of pixel membership to the existing objects to classify pixels as the edge or object. Six entropy-based algorithms are evaluated, i.e., the optimum Entropy based on Shannon formula, the optimum of relative-entropy based on Kullback-Leibler divergence, the maximum of optimum entropy neighbor, the minimum of optimum relative-entropy neighbor, the thinning of optimum entropy neighbor, and the thinning of optimum relative-entropy neighbor. The experiment is held to compare the developed algorithms against Canny as a benchmark by employing five performance parameters, i.e., the average number of detected objects, the average number of detected edge pixels, the average size of detected objects, the ratio of the number of edge pixel per object, and the average of ten biggest sizes. The experiment shows that the entropy-based algorithms significantly improve the production of closed-loop edges, and the optimum of relative-entropy neighbor based on Kullback-Leibler divergence becomes the most desired approach among others due to the production of more considerable closed-loop edge in the average. This finding suggests that the entropy-based algorithm is the best choice for edge-based segmentation. The effectiveness of Entropy in the segmentation task is addressed for further research.

EEG complexity-based algorithm using Multiscale Fuzzy Entropy: Towards a detection of Alzheimer’s disease

Alzheimer’s Disease (AD) is a progressive neurodegenerative condition causing memory, attention, and language decline. Current AD diagnostic methods lack objectivity and non-invasiveness. While electroencephalography (EEG) holds promise for AD research, conventional EEG analysis methods have proven unsatisfactory. Non-linear dynamical approaches are considered more effective in assessing the brain’s complex nature. Starting from these considerations, this study presents an entropy-based algorithm utilizing Multiscale Fuzzy Entropy (MFE) as a promising, effective AD diagnostic method. Computed across 20 different time scales for a public dataset, MFE showed a significant discriminative power. Notably, a trend inversion was observed in the results: AD subjects displayed higher complexity values for slow frequency bands compared to healthy controls, while the opposite was found in fast frequency bands. These findings underscore the potential of MFE in effectively distinguishing AD patients from healthy individuals, marking a significant advance towards more objective and reliable AD diagnosis strategies.

Detection and Mitigation of Malicious DDoS Floods in Software Defined Networks

The advent of software-defined networking (SDN) has significantly transformed network management by offering modular control and data plane characteristics, enabling adaptability and flexibility in managing networks. This innovation entails the separation of control and data plane elements to facilitate efficient network administration. Nevertheless, the centralization resulting from control plane separation renders SDN vulnerable to cyber threats, particularly Distributed Denial-of-service (DDoS) attacks that target SDN controllers. Recently, studies have highlighted the relevance of entropy-based attack detection techniques compared to alternative methods. However, relying solely on entropy may overlook detection in specific variables, such as flow specification variations. To address the limitations of entropy-based detection systems, we developed a DDoS attack detection framework within the SDN control plane, integrating the packet flow initiation and specification properties with an entropy-based algorithm to ensure accurate attack detection measures. Our lightweight framework aims to mitigate DDoS attacks by detecting their impact in the early stages, thus preventing SDN controllers from being hijacked due to excessive packet flooding. The simulation is employed in Mininet network simulator to implement, and the testbed is created by focusing UDP flood attacks in widely used data-centric tree topologies. The experimental results demonstrate that our proposed solution effectively detects and mitigates novel parameters of SDN-based DDoS floods within 150 packets while maintaining minimal delay and high accuracy

Generalized minimum error entropy Kalman filter for non-Gaussian noise

Error entropy is a well-known learning criterion in information theoretic learning (ITL), and it has already been applied to a wide range of fields. However, the shape of error entropy cannot be changed freely since its kernel function is the Gaussian kernel function, which causes the error entropy-based algorithm to handle only some specific kinds of noises. Benefiting from the property that the generalized Gaussian kernel function is free to adjust its shape, a novel Kalman-type filter algorithm based on the generalized minimum error entropy (GMEEKF) criterion is derived. Moreover, the mean error behavior, mean square error behavior, and computational complexity of the GMEEKF algorithm are analyzed. Finally, several simulations and experiments are performed to demonstrate the performance of the GMEEKF algorithm in comparison with the existing Kalman-type filter algorithms.

System Identification of an Actuated Inclined Ball Mechanism Via Causation Entropy

Abstract Grey-box and black-box dynamic models are routinely used to model the behavior of real-world dynamic systems. When creating such models, the identification of an accurate model structure (often referred to as covariate selection, feature selection, or sparsity identification) is a critical step required to achieve suitable predictive performance by minimizing the effects of overfitting. Recently, causation entropy has been shown to be quite useful in data-driven covariate selection as it provides a mechanism to measure the causal relationships between the set of covariates and the state dynamics. This work extends previous results by applying the causation entropy covariate selection technique to data from an experimental nonlinear system consisting of a ball rolling on an actuated inclined ramp. Data collected from the system is processed by the causation entropy-based algorithm and covariate selection is performed on a black-box dynamic model. The resulting optimized model is shown to provide better predictive performance than an optimized black-box model which includes extraneous covariates. This study represents the first application of causation entropy-based covariate selection to real-world experimental data, illustrating its use as a practical system identification method.

Semantic Image Segmentation Using Scant Pixel Annotations

The success of deep networks for the semantic segmentation of images is limited by the availability of annotated training data. The manual annotation of images for segmentation is a tedious and time-consuming task that often requires sophisticated users with significant domain expertise to create high-quality annotations over hundreds of images. In this paper, we propose the segmentation with scant pixel annotations (SSPA) approach to generate high-performing segmentation models using a scant set of expert annotated images. The models are generated by training them on images with automatically generated pseudo-labels along with a scant set of expert annotated images selected using an entropy-based algorithm. For each chosen image, experts are directed to assign labels to a particular group of pixels, while a set of replacement rules that leverage the patterns learned by the model is used to automatically assign labels to the remaining pixels. The SSPA approach integrates active learning and semi-supervised learning with pseudo-labels, where expert annotations are not essential but generated on demand. Extensive experiments on bio-medical and biofilm datasets show that the SSPA approach achieves state-of-the-art performance with less than 5% cumulative annotation of the pixels of the training data by the experts.

Optimum seismic design of curved cable-stayed bridges

One of the current advances in cable-stayed bridges design is the ability to create curved geometries to fulfill architectural requirements or site constraints. There are currently 68 curved cable-stayed bridges under operation, 56 of which have been completed in the current millennium. Most of them are footbridges, but the number of road and railway applications is increasing. However, there is not much information concerning the design of these structures. This paper concerns a 350 m long curved cable-stayed road bridge with a special emphasis on the seismic optimum design. The tower-deck connection is a key aspect for the seismic response. The longitudinal movement between tower and deck is released, the transverse and vertical support stiffness and damping are considered as design variables. For this structural-control optimization, linear viscous dampers are sought. The spatial variability of the seismic ground motion and construction erection stages for curved CSB are considered. An entropy-based algorithm is used to find the optimum solutions for two designs, S1 where the seismic loading is scaled to 30% and S2 with full seismic intensity. The results show that optimum design S2 costs 21.5% more than S1 optimum and 43% more than a similar straight bridge.

Heterogeneity Analysis of Bladder Cancer Based on DNA Methylation Molecular Profiling.

Bladder cancer is a highly complex and heterogeneous malignancy. Tumor heterogeneity is a barrier to effective diagnosis and treatment of bladder cancer. Human carcinogenesis is closely related to abnormal gene expression, and DNA methylation is an important regulatory factor of gene expression. Therefore, it is of great significance for bladder cancer research to characterize tumor heterogeneity by integrating genetic and epigenetic characteristics. This study explored specific molecular subtypes based on DNA methylation status and identified subtype-specific characteristics using patient samples from the TCGA database with DNA methylation and gene expression were measured simultaneously. The results were validated using an independent cohort from GEO database. Four DNA methylation molecular subtypes of bladder cancer were obtained with different prognostic states. In addition, subtype-specific DNA methylation markers were identified using an information entropy-based algorithm to represent the unique molecular characteristics of the subtype and verified in the test set. The results of this study can provide an important reference for clinicians to make treatment decisions.

Morphological geodesic active contour algorithm for the segmentation of the histogram‐equalized welding bead image edges

Assessment and evaluation are the essential processes of industrially manufactured products for the determination of the quality and quantity of products. They give justifications in a practical way about whether the machine is perfect or imperfect, which can lead to a better or poorer production. In this study, the authors propose an algorithm that uses morphological geodesic active contour and image processing techniques to perform segmentation and assess the performance of a robot used to manufacture welding beads. The algorithm has four parameters which are pre-processed images, balloon force, smoothing parameter, and number of iterations. To pre-process the images, the algorithm uses an inverse Gaussian gradient operator for edge detection and applies the histogram equalization method to level the distribution. To detect the external contour of the bead, the level set is initialized as the region of interest whereby a balloon force can inflate or deflate towards the edges. To smoothen the contour, a smoothing parameter is applied to convert the jagged lines into a curve over a reasonable number of iterations. Based on the experimental results, the authors’ algorithm used a fixed balloon force of −2, a smoothing parameter value of 4, and 40 iterations to segment images obtained from three different environments. The computation time for the segmentation and evaluation of one image was 0.70, 0.61, and 0.67 s for datasets with high brightness, low brightness, and normal brightness, respectively. Additionally, the authors’ proposed algorithm achieved an outstanding performance of 0.9954, 0.9843, 0.9892, and 0.9435 in terms of recall, precision, F-measure, and IOU, respectively. To justify the performance of the authors’ proposed algorithm, the authors compared it with the existing algorithms and found that it worked better than all the others for segmentation, although it lagged behind the entropy-based algorithm in terms of speed.

Entropy-Based Anomaly Detection in Household Electricity Consumption

Energy efficiency is one of the most important current challenges, and its impact at a global level is considerable. To solve current challenges, it is critical that consumers are able to control their energy consumption. In this paper, we propose using a time series of window-based entropy to detect anomalies in the electricity consumption of a household when the pattern of consumption behavior exhibits a change. We compare the accuracy of this approach with two machine learning approaches, random forest and neural networks, and with a statistical approach, the ARIMA model. We study whether these approaches detect the same anomalous periods. These different techniques have been evaluated using a real dataset obtained from different households with different consumption profiles from the Madrid Region. The entropy-based algorithm detects more days classified as anomalous according to context information compared to the other algorithms. This approach has the advantages that it does not require a training period and that it adapts dynamically to changes, except in vacation periods when consumption drops drastically and requires some time for adapting to the new situation.

A Novel Radiomics-Based Tumor Volume Segmentation Algorithm for Lung Tumors in FDG-PET/CT after 3D Motion Correction-A Technical Feasibility and Stability Study.

Positron emission tomography (PET) provides important additional information when applied in radiation therapy treatment planning. However, the optimal way to define tumors in PET images is still undetermined. As radiomics features are gaining more and more importance in PET image interpretation as well, we aimed to use textural features for an optimal differentiation between tumoral tissue and surrounding tissue to segment-target lesions based on three textural parameters found to be suitable in previous analysis (Kurtosis, Local Entropy and Long Zone Emphasis). Intended for use in radiation therapy planning, this algorithm was combined with a previously described motion-correction algorithm and validated in phantom data. In addition, feasibility was shown in five patients. The algorithms provided sufficient results for phantom and patient data. The stability of the results was analyzed in 20 consecutive measurements of phantom data. Results for textural feature-based algorithms were slightly worse than those of the threshold-based reference algorithm (mean standard deviation 1.2%—compared to 4.2% to 8.6%) However, the Entropy-based algorithm came the closest to the real volume of the phantom sphere of 6 ccm with a mean measured volume of 26.5 ccm. The threshold-based algorithm found a mean volume of 25.0 ccm. In conclusion, we showed a novel, radiomics-based tumor segmentation algorithm in FDG-PET with promising results in phantom studies concerning recovered lesion volume and reasonable results in stability in consecutive measurements. Segmentation based on Entropy was the most precise in comparison with sphere volume but showed the worst stability in consecutive measurements. Despite these promising results, further studies with larger patient cohorts and histopathological standards need to be performed for further validation of the presented algorithms and their applicability in clinical routines. In addition, their application in other tumor entities needs to be studied.

SEPA: signaling entropy-based algorithm to evaluate personalized pathway activation for survival analysis on pan-cancer data.

Biomarkers with prognostic ability and biological interpretability can be used to support decision-making in the survival analysis. Genes usually form functional modules to play synergistic roles, such as pathways. Predicting significant features from the functional level can effectively reduce the adverse effects of heterogeneity and obtain more reproducible and interpretable biomarkers. Personalized pathway activation inference can quantify the dysregulation of essential pathways involved in the initiation and progression of cancers, and can contribute to the development of personalized medical treatments. In this study, we propose a novel method to evaluate personalized pathway activation based on signaling entropy for survival analysis (SEPA), which is a new attempt to introduce the information-theoretic entropy in generating pathway representation for each patient. SEPA effectively integrates pathway-level information into gene expression data, converting the high-dimensional gene expression data into the low-dimensional biological pathway activation scores. SEPA shows its classification power on the prognostic pan-cancer genomic data, and the potential pathway markers identified based on SEPA have statistical significance in the discrimination of high- and low-risk cohorts and are likely to be associated with the initiation and progress of cancers. The results show that SEPA scores can be used as an indicator to precisely distinguish cancer patients with different clinical outcomes, and identify important pathway features with strong discriminative power and biological interpretability. The MATLAB-package for SEPA is freely available from https://github.com/xingyili/SEPA. Supplementary data are available at Bioinformatics online.

A Multi-objective Cross Entropy-based algorithm for sustainable global food supply chain with risk considerations: A case study

This paper proposes a sustainable global food supply chain model with risk considerations. Perishable foods begin to deteriorate after were produced. Therefore, without proper maintenance and transportation, the inventories may deteriorate quickly before be used which leads to unavoidable costs. The proposed model has four levels; suppliers, producers, warehouses and demand centers. The sustainable supply chain has three objectives, the first and second one is to reduce economic costs and environmental impacts, respectively and the third one is to maximize the level of employment in the chain. Also, because of nature of the supply chain, several risk factors are investigated in the model including: time delay risk, exchange rate fluctuations, raw material quality and production quality. For solving large instances, a Multi-Objective Cross Entropy (CE) algorithm is introduced. The proposed CE method is efficient in terms of its run time and low difference with the GAMS software. The MID and SM criteria which show the dispersion of the solutions are lower by CE method than the Epsilon constraint method. Also, a robust optimization approach is introduced to overcome the uncertainty. Also, a validation method is performed to evaluate the effectiveness of the robust approach. Then Sensitivity analysis is done to show capability of the model. Finally, the problem is applied for a food company in Iran and the city of Mashhad. The results show the establishment of a new factory in Qazvin lead to increase 2.8% of total costs, whereas decreasing 27.4% of CO2 emissions and increasing 44.2% of social benefits is resulted.

A jackknife entropy-based clustering algorithm for probability density functions

This paper proposes a new unsupervised learning algorithm called jackknife entropy-based clustering algorithm for grouping families of probability density functions (pdfs). The fitness function is used to choose the best threshold values of similarity in the proposed algorithm. We demonstrate the correctness and robustness of the proposed algorithm on a synthetic data set. Finally, we apply the algorithm to texture clustering.

Resisting HODPA attacks in modular exponentiation using inner product with differential evolution

This paper presents a secured computation of modular exponentiation to resist higher-order differential power analysis (HODPA) attacks in asymmetric cryptosystems like RSA. HODPA attacks can be resisted by segmenting secret sensitive data and its intermediate values into multiple shares. In modular exponentiation-based cryptosystems, the exponent plays a significant part in the secret key. We have used inner product with differential evolution algorithm to segment the exponent into multiple shares. Using entropy-based nearest neighbor algorithm, we have randomly computed independent modular exponentiation to resist SPA and DPA attacks. Analysis was done on 1024, 1536 and 2048 bit RSA. With a pre-computation complexity, the proposed approach can provide significant resistance to SPA, DPA and HODPA attacks against modular exponentiation-based cryptosystems.

Optimization of Entropy-Based Automated Dyslalia Screening Algorithm.

This paper presents the current state of progress of a project aimed at achieving an automated information entropy-based discrimination of phoneme mispronunciations in utterances of early school-age children. The introductory part briefly describes the dyslalia symptomology and the incidence of dyslalic disorders. This section also reviews the current challenges posed by the main research objective in other similar projects sharing the same objective and summarizes the current results thereof. The Material and Method section presents the conditions, the technology and the feature-extraction technique used in the experiment. The same section also describes the computation of the information entropy values of each analyzed speech sample. The highest match rate of 93.33% was achieved in the classification of words containing the phoneme /r/ in the initial position. A synthesis of the achieved results is provided in the Results section based on which conclusions are drawn and exposed in the Discussion and Conclusions section.