Symmetric Cross Entropy Research Articles

Hierarchical symmetric cross entropy for distant supervised relation extraction

Hierarchical symmetric cross entropy for distant supervised relation extraction

Taming over-smoothing representation on heterophilic graphs

Graph Neural Networks (GNNs) have shown expressive modeling ability on graphs. An inherent assumption is that those connected graph nodes exhibit strong homophily, passing consistent label information into the associated central node representation. However, heterophilic graphs may hold inconsistent labels for those connected nodes, degenerating the performance of the typical message passing and exacerbating its over-smoothing issue. In this paper, to tame heterophilic graphs for GNNs, a novel label estimation-based message passing scheme is presented, which can further relieve the over-smoothing and lead the learned representations to be more distinguishable on graphs with either homophily or heterophily. Specifically, we optimize the edge weights of the graph by invoking the label estimation with the goal of returning a more effective adjacency matrix. With this matrix, a novel message passing scheme is presented, aggregating neighbor information with consistent labels to their central nodes. Meanwhile, a symmetric cross entropy is also employed to mitigate negative effects from noisy labels. With such guarantees in our new message passing, a uniform framework termed Label Estimation-based GNN (LE-GNN) is finally established. Extensive experiments demonstrate the ability of our model to prevent over-smoothing and show its effectiveness for node classification on both homophilic and heterophilic graphs.

Robust Malware identification via deep temporal convolutional network with symmetric cross entropy learning

AbstractRecent developments in the field of Internet of things (IoT) have aroused growing attention to the security of smart devices. Specifically, there is an increasing number of malicious software (Malware) on IoT systems. Nowadays, researchers have made many efforts concerning supervised machine learning methods to identify malicious attacks. High‐quality labels are of great importance for supervised machine learning, but noises widely exist due to the non‐deterministic production environment. Therefore, learning from noisy labels is significant for machine learning‐enabled Malware identification. In this study, motivated by the symmetric cross entropy with satisfactory noise robustness, the authors propose a robust Malware identification method using temporal convolutional network (TCN). Moreover, word embedding techniques are generally utilised to understand the contextual relationship between the input operation code (opcode) and application programming interface function names. Here, considering the numerous unlabelled samples in real‐world intelligent environments, the authors pre‐train the TCN model on an unlabelled set using a word embedding method, that is, Word2Vec. In the experiments, the proposed method is compared with several traditional statistical methods and more recent neural networks on a synthetic Malware dataset and a real‐world dataset. The performance comparisons demonstrate the better performance and noise robustness of their proposed method, especially that the proposed method can yield the best identification accuracy of 98.75% in real‐world scenarios.

Symmetric cross-entropy multi-threshold color image segmentation based on improved pelican optimization algorithm.

To address the problems of low accuracy and slow convergence of traditional multilevel image segmentation methods, a symmetric cross-entropy multilevel thresholding image segmentation method (MSIPOA) with multi-strategy improved pelican optimization algorithm is proposed for global optimization and image segmentation tasks. First, Sine chaotic mapping is used to improve the quality and distribution uniformity of the initial population. A spiral search mechanism incorporating a sine cosine optimization algorithm improves the algorithm's search diversity, local pioneering ability, and convergence accuracy. A levy flight strategy further improves the algorithm's ability to jump out of local minima. In this paper, 12 benchmark test functions and 8 other newer swarm intelligence algorithms are compared in terms of convergence speed and convergence accuracy to evaluate the performance of the MSIPOA algorithm. By non-parametric statistical analysis, MSIPOA shows a greater superiority over other optimization algorithms. The MSIPOA algorithm is then experimented with symmetric cross-entropy multilevel threshold image segmentation, and eight images from BSDS300 are selected as the test set to evaluate MSIPOA. According to different performance metrics and Fridman test, MSIPOA algorithm outperforms similar algorithms in global optimization and image segmentation, and the symmetric cross entropy of MSIPOA algorithm for multilevel thresholding image segmentation method can be effectively applied to multilevel thresholding image segmentation tasks.

Fault Diagnosis Algorithm of Gearboxes Based on GWO-SCE Adaptive Multi-Threshold Segmentation and Subdomain Adaptation

The data distribution of the vibration signal under different speed conditions of the gearbox is different, which leads to reduced accuracy of fault diagnosis. In this regard, this paper proposes a deep transfer fault diagnosis algorithm combining adaptive multi-threshold segmentation and subdomain adaptation. First of all, in the data acquisition stage, a non-contact, easy-to-arrange, and low-cost sound pressure sensor is used to collect equipment signals, which effectively solves the problems of contact installation limitations and increasingly strict layout requirements faced by traditional vibration signal-based methods. The continuous wavelet transform (CWT) is then used to convert the original vibration signal of the device into time–frequency image samples. Further, to highlight the target fault characteristics of the samples, the gray wolf optimization algorithm (GWO) is combined with symmetric cross entropy (SCE) to perform adaptive multi-threshold segmentation on the image samples. A convolutional neural network (CNN) is then used to extract the common features of the source domain samples and the target domain samples. Additionally, the local maximum mean discrepancy (LMMD) is introduced into the parameter space of the deep fully connected layer of the network to align the sub-field edge distribution of deep features so as to reduce the distribution difference of sub-class fault features under different working conditions and improve the diagnostic accuracy of the model. Finally, to verify the effectiveness of the proposed diagnosis method, a fault preset experiment of the gearbox under variable speed conditions is carried out. The results show that compared to other diagnostic methods, the method in this paper has higher diagnostic accuracy and superiority.

Multi-Level Thresholding Image Segmentation Based on Improved Slime Mould Algorithm and Symmetric Cross-Entropy.

Multi-level thresholding image segmentation divides an image into multiple regions of interest and is a key step in image processing and image analysis. Aiming toward the problems of the low segmentation accuracy and slow convergence speed of traditional multi-level threshold image segmentation methods, in this paper, we present multi-level thresholding image segmentation based on an improved slime mould algorithm (ISMA) and symmetric cross-entropy for global optimization and image segmentation tasks. First, elite opposition-based learning (EOBL) was used to improve the quality and diversity of the initial population and accelerate the convergence speed. The adaptive probability threshold was used to adjust the selection probability of the slime mould to enhance the ability of the algorithm to jump out of the local optimum. The historical leader strategy, which selects the optimal historical information as the leader for the position update, was found to improve the convergence accuracy. Subsequently, 14 benchmark functions were used to evaluate the performance of ISMA, comparing it with other well-known algorithms in terms of the optimization accuracy, convergence speed, and significant differences. Subsequently, we tested the segmentation quality of the method proposed in this paper on eight grayscale images and compared it with other image segmentation criteria and well-known algorithms. The experimental metrics include the average fitness (mean), standard deviation (std), peak signal to noise ratio (PSNR), structure similarity index (SSIM), and feature similarity index (FSIM), which we utilized to evaluate the quality of the segmentation. The experimental results demonstrated that the improved slime mould algorithm is superior to the other compared algorithms, and multi-level thresholding image segmentation based on the improved slime mould algorithm and symmetric cross-entropy can be effectively applied to the task of multi-level threshold image segmentation.

Optimizing Multiple Entropy Thresholding by the Chaotic Combination Strategy Sparrow Search Algorithm for Aggregate Image Segmentation.

Aggregate measurement and analysis are critical for civil engineering. Multiple entropy thresholding (MET) is inefficient, and the accuracy of related optimization strategies is unsatisfactory, which results in the segmented aggregate images lacking many surface roughness and aggregate edge features. Thus, this research proposes an autonomous segmentation model (i.e., PERSSA-MET) that optimizes MET based on the chaotic combination strategy sparrow search algorithm (SSA). First, aiming at the characteristics of the many extreme values of an aggregate image, a novel expansion parameter and range-control elite mutation strategies were studied and combined with piecewise mapping, named PERSSA, to improve the SSA's accuracy. This was compared with seven optimization algorithms using benchmark function experiments and a Wilcoxon rank-sum test, and the PERSSA's superiority was proved with the tests. Then, PERSSA was utilized to swiftly determine MET thresholds, and the METs were the Renyi entropy, symmetric cross entropy, and Kapur entropy. In the segmentation experiments of the aggregate images, it was proven that PERSSA-MET effectively segmented more details. Compared with SSA-MET, it achieved 28.90%, 12.55%, and 6.00% improvements in the peak signal-to-noise ratio (PSNR), the structural similarity (SSIM), and the feature similarity (FSIM). Finally, a new parameter, overall merit weight proportion (OMWP), is suggested to calculate this segmentation method's superiority over all other algorithms. The results show that PERSSA-Renyi entropy outperforms well, and it can effectively keep the aggregate surface texture features and attain a balance between accuracy and speed.

Forestry Canopy Image Segmentation Based on Improved Tuna Swarm Optimization

Forests play a vital role in increasing carbon sequestration in the biosphere. In recent years, segmenting forest canopy images in order to obtain various plant population parameters has become an essential means to assess the ecosystem. The objective of forest canopy image segmentation is to separate and extract sky regions from the background. This study proposes a hybrid method based on improved tuna swarm optimization (ITSO) for forestry canopy image segmentation. The symmetric cross-entropy is introduced to perform forestry canopy image thresholding by modeling the classes of an image as membership functions. In order to achieve the optimal thresholds of the forest canopy image, the entropy-solving procedure is arduous and time-consuming. In order to resolve this issue, the ITSO method was adopted to search for the most significant threshold. Meanwhile, the Tent chaotic map is used to initialize the tuna population according to the chaotic factor. The experiment is carried out on four different types of forest canopy images, with four indices (MAE, RVD, IoU, and ASD) used for quantitative analysis. The experiment’s results show that the ITSO-based segmentation method outperforms others, making it a better way to segment images of forest canopies.

Model-based temporal evolution and spatial equilibrium analysis of green development in China's Yangtze River Economic Belt from 2009 to 2018

Green development is one of the important guidance and core contents of high-quality development of regional economy and society, and its spatial–temporal pattern evolution analysis is helpful to the scientific formulation of relevant policies. However, the green development evaluation is faced with multi attribute decision making difficulties. In view of this, firstly, based on discussing the connotation of green development from ontology, epistemology, axiology and methodology, a new evaluation indicator system from the four dimensions of energy conservation, emission reduction, efficiency increase and harmony was explored. Then, coupling symmetric cross entropy multi attribute decision making and Gini coefficient method, a comprehensive evaluation framework of temporal evolution and spatial equilibrium of green development was constructed and applied in 11 provinces (cities) of China's Yangtze River Economic Belt (YEB) from 2009 to 2018. The results show that during the study period, the green development level of YEB has improved year by year, but there was still a large gap. Among the 120 research samples, only 41.7% had a ranking score of more than 6, and only Chongqing in 2017 had a ranking score of more than 8. The Gini coefficient of green development in 11 provinces (cities) is about 0.2, which is balanced on the whole, but there is a significant difference in the indicators, such as proportion of new energy power generation, sulfur dioxide emission per 104 RMB yuan GDP and proportion of investment in environmental protection; The effect of energy conservation and emission reduction is obvious, while efficiency increase and harmony need to be improved. The complementary role of 11 provinces (cities) in YEB's green development has become increasingly prominent, which made that YEB (as a whole) ranked second in comparison with 11 provinces (cities) in 2018.

Green Logistics Partner Selection Based on Pythagorean Hesitant Fuzzy Set and Multiobjective Optimization

With the deepening of the uncertainty and fuzziness of people’s understanding of things, it becomes more convenient to use fuzzy decision making in evaluating schemes. These schemes not only include quantitative data but also consider more and more uncertainty evaluation information. In fuzzy theory, the Pythagorean fuzzy set is extended, both of which are less than or equal to 1, which increases the scope of application. This paper mainly studies the selection of green logistics. Based on the original TOPSIS principle, this paper introduces a Pythagorean fuzzy symmetric cross entropy to describe the difference between two Pythagorean fuzzy numbers and proves its rationality. The model is applied to the selection of green logistics and compared with the results obtained by other different methods, which also shows the effectiveness and practicability of the model. It extends the use of fuzzy sets and can solve some problems that cannot be solved by intuitionistic fuzzy sets before.

Hybrid MCDM Based on VIKOR and Cross Entropy under Rough Neutrosophic Set Theory

Problems in real life usually involve uncertain, inconsistent and incomplete information. An example of such problems is strategic decision making with respect to remediation planning of historic pedestrian bridges. The multiple decision makers and experts, as well as the various mutually conflicting criteria, unknown criteria weights, and vagueness and duality in final decisions, provide motivation to develop a methodology that is able to resist the challenges implicit in this problem. Therefore, the aim of this research was to propose an algorithm based on the theory of rough neutrosophic sets in order to solve the problem of strategic planning with respect to the remediation of historic pedestrian bridges. A new multicriteria decision-making model is developed that is a fusion of rough set and neutrosophic set theory. A new cross entropy is proposed under a rough neutrosophic environment that does not possess the shortcomings of asymmetrical character and unknown occurrences. Additionally, a weighted rough neutrosophic symmetric cross entropy is proposed. Furthermore, a rough neutrosophic VIKOR method is introduced, with which the values of the utility measure, regret measure and VIKOR index are obtained. These values, as well as the weighted rough neutrosophic symmetric cross entropy measure, are used to provide a ranking of historic pedestrian bridges favorable to remediation. Finally, an illustrative example of the strategic planning of remediation for historic pedestrian bridges is solved and compared to other research, demonstrating the robustness, feasibility and efficacy of the model when dealing with complex multicriteria decision-making processes.

Adaptive Hierarchical Probabilistic Model Using Structured Variational Inference for Point Set Registration

Point set registration plays an important role in computer vision and pattern recognition. In this article, we propose an adaptive hierarchical probabilistic model (HPM) under a variational Bayesian (VB) framework for point set registration problem. The main contributions of this article are given as follows. First, a dynamic putative inlier estimation strategy is proposed through the hesitant fuzzy Einstein weighted averaging based membership calculation and component estimation using symmetric cross entropy. Second, a student-t mixture model based HPM is designed to solve outlier and occlusion problems during registration. Third, a VB-based transformation updating is proposed to construct a robust and adjustable transformation for effectively fitting target point set while further resisting outliers. The performances of the proposed method in point set and image registrations against 11 state-of-the-art methods are evaluated, in which our method gives the best performance in most scenarios.

RSF model with SCE‐based global constraint for image segmentation

A novel region-scalable fitting (RSF) active contour model with symmetric cross entropy (SCE)-based global constraint is proposed to segment various kinds of images. The energy functional is mainly composed of two parts, namely the local and global constraints. The local constraint is the weighted region-scalable fitting term, which computes the local intensity information. The global constraint is constructed by measuring the difference between the intensity distribution of the object image and the intensity distribution of the original image using SCE. Moreover, some regularised terms are incorporated into the energy functional. Experimental results for synthetic and real images show that the proposed model achieves better segmentation performance and is insensitive to initial contour and noise.

Fault diagnosis of rolling element bearing based on symmetric cross entropy of neutrosophic sets

A novel symmetric single-valued neutrosophic cross entropy (SVNCE) measure based upon a newly developed symmetric measure of fuzzy cross entropy is established and then applied it for identifying defects of bearings installed in a test rig and axial piston pump. A 3-level wavelet packet transformation (WPT) is used for extracting and decomposing the fault features associated with the vibrational signals. The new evaluated SVNCE values between knowledge of various faults types and real testing samples provide useful evaluation information of fault types. The concluded results reveal that the proposed SVNCE measure furnishes better fault identification accuracy when compared with the existing model based upon correlation coefficient of simplified neutrosophic sets (SNSs). Furthermore, the proposed SVNCE measure offers consistent and feasible results and is capable for holding the optimal fault type selection under sensitive analysis.

A novel active contour model based on modified symmetric cross entropy for remote sensing river image segmentation

The traditional active contour models cannot segment the remote sensing river images accurately. To solve this problem, a novel active contour model based on modified symmetric cross entropy is proposed. In the proposed model, the external energy constraint terms are defined by the symmetric cross entropy and the region fitting centers are represented by the medians of pixel grayscale values inside the object and background regions. Moreover, the penalty energy term is incorporated into the energy functional to avoid the re-initialization. In order to improve the segmentation efficiency of the proposed model, the Chebyshev distance between the pixel grayscale values inside the region and its region fitting center is chosen as its region energy weight, which can be adaptively adjusted, instead of the constant region energy weight. The extensive experiments are performed on a large number of remote sensing river images and the results demonstrate that, compared with the CV model, the GAC model, the CEACM model, the RSF model, the LIF model, and the LGIF model, the proposed model can segment the images more accurately and rapidly, which has the clear advantages in both segmentation performance and segmentation efficiency.

A novel cross-entropy and entropy measures of IFSs and their applications

In this paper we discussed novel cross-entropy and entropy models on intuitionism and fuzziness of intuitionistic fuzzy sets (IFSs). In order to measure the discrimination uncertain information, cross-entropy and symmetric cross-entropy are defined based on intuitionistic factor and fuzzy factor. In particular, the constructive principle of entropy is refined; relationship between cross-entropy and entropy is investigated, so we establish a novel intuitionistic fuzzy entropy formula that simultaneously takes into account intuitionistic entropy and fuzzy entropy. We also propose concepts of marginal rate of substitution (MRS) and elasticity of substitution (ES) to describe substitution effect between intuitionism and fuzziness. Moreover, we extend the cross-entropy and entropy models to general version, which could be adjusted by different parameters, and comparative analysis with other existed cross-entropy and entropy measures are presented. Finally, we give applications of pattern recognition and decision making to demonstrate the efficiency of the cross-entropy and entropy models.

Some inequalities relating different measures of divergence between two probability distributions

This note presents new inequalities relating different divergence measures in the family of "convex likelihood-ratio expectation" measures of Csiszar (1967), Ali and Silvey (1966), and especially in the single-parameter family of "AM-GM" divergence measures. The most prominent result is that /spl theta//sup 2//spl les/ 1/4 J, where /spl theta/ is the Bhattacharyya angle of divergence (a true distance metric), and J is the symmetric cross-entropy. A pair of "log /spl Gamma/" divergences is also introduced and related to the cross-entropies I and J.