Initial Partition Research Articles

Parallel power load abnormalities detection using fast density peak clustering with a hybrid canopy-K-means algorithm

Parallel power loads anomalies are processed by a fast-density peak clustering technique that capitalizes on the hybrid strengths of Canopy and K-means algorithms all within Apache Mahout’s distributed machine-learning environment. The study taps into Apache Hadoop’s robust tools for data storage and processing, including HDFS and MapReduce, to effectively manage and analyze big data challenges. The preprocessing phase utilizes Canopy clustering to expedite the initial partitioning of data points, which are subsequently refined by K-means to enhance clustering performance. Experimental results confirm that incorporating the Canopy as an initial step markedly reduces the computational effort to process the vast quantity of parallel power load abnormalities. The Canopy clustering approach, enabled by distributed machine learning through Apache Mahout, is utilized as a preprocessing step within the K-means clustering technique. The hybrid algorithm was implemented to minimise the length of time needed to address the massive scale of the detected parallel power load abnormalities. Data vectors are generated based on the time needed, sequential and parallel candidate feature data are obtained, and the data rate is combined. After classifying the time set using the canopy with the K-means algorithm and the vector representation weighted by factors, the clustering impact is assessed using purity, precision, recall, and F value. The results showed that using canopy as a preprocessing step cut the time it proceeds to deal with the significant number of power load abnormalities found in parallel using a fast density peak dataset and the time it proceeds for the k-means algorithm to run. Additionally, tests demonstrate that combining canopy and the K-means algorithm to analyze data performs consistently and dependably on the Hadoop platform and has a clustering result that offers a scalable and effective solution for power system monitoring.

Robustness of community structure under edge addition.

Communities often represent key structural and functional clusters in networks. To preserve such communities, it is important to understand their robustness under network perturbations. Previous work in community robustness analysis has focused on studying changes in the community structure as a response of edge rewiring and node or edge removal. However, the impact of increasing connectivity on the robustness of communities in networked systems is relatively unexplored. Studying the limits of community robustness under edge addition is crucial to better understanding the cases in which density expands or false edges erroneously appear. In this paper, we analyze the effect of edge addition on community robustness in synthetic and empirical temporal networks. We study two scenarios of edge addition: random and targeted. We use four community detection algorithms, Infomap, Label Propagation, Leiden, and Louvain, and demonstrate the results in community similarity metrics. The experiments on synthetic networks show that communities are more robust when the initial partition is stronger or the edge addition is random, and the experiments on empirical data also indicate that robustness performance can be affected by the community similarity metric. Overall, our results suggest that the communities identified by the different types of community detection algorithms exhibit different levels of robustness, and so the robustness of communities depends strongly on the choice of detection method.

Dynamic and heterogeneity-sensitive urban network partitioning: a data-driven technique

Information of Connected Vehicles (CVs) could describe vehicular dynamics in much greater detail, enhancing the effectiveness of traffic control systems. One important such system is perimeter control, which can achieve better performance by incorporating the evolution of congestion into the identification of protected regions through a dynamic approach. However, little attention has been given to identifying such dynamic regions by developing CV-based network partitioning models in a spatiotemporal dimension. To address this gap, this paper proposes a three-module framework that (1) collects the relevant information of CVs, (2) performs initial partitioning based on some rational considerations, and (3) identifies the optimal protected regions through a partitioning evaluation, improvement, and iteration algorithm. The carried-out comparisons between perimeter control systems employing the resulting protected regions and those using static regions confirm that the proposed framework enhances the efficiency of perimeter control, even for CVs' penetration rates that are as low as 15%.

Research on Road Network Partitioning Considering the Coupling of Network Connectivity and Traffic Attributes

The urban road network is a large and complex system characterized by significant heterogeneity arising from different spatial structures and traffic demands. To facilitate effective management and control, it is necessary to partition the road network into homogeneous sub-areas. In this regard, we aim to propose a hybrid method for partitioning sub-areas with intra-area homogeneity, inter-area heterogeneity, and similar sizes, called CSDRA. It is specifically designed for bidirectional road networks with segment weights that encompass traffic flow, speed, or roadside facility evaluation. Based on community detection and spectral clustering, this proposed method comprises four main modules: initial partition, partitioning of large sub-areas, reassignment of small sub-areas, and boundary adjustment. In the preliminary partitioning work, we also design a road network reconstruction method which further helps to enhance the intra-area homogeneity and inter-area heterogeneity of partitioning results. Furthermore, to align with the requirement for comparable work units in practical traffic management and control, we control the similarity in the size of sub-areas by enforcing upper and lower bound constraints on the size of the sub-areas. We verify the outperformance of the proposed method by an experiment on the partitioning of an urban road network in Guangzhou, China, where we employ sidewalk barrier-free score data as segment weights. The results demonstrate the effectiveness of both the road network reconstruction method and the CSDRA proposed in this paper, as they significantly improve the partitioning outcomes compared with other methods using different evaluation indicators corresponding to the partitioning objectives. Finally, we investigate the influence of constraint parameters on the evaluation indicator. Our findings indicate that appropriately configuring these constraint parameters can effectively minimize sub-region size variations while having minimal impact on other aspects.

Cluster partition for distributed energy resources in Regional Integrated Energy System

Fully utilizing local clean energy by regional Integrated Energy System is a effective way to alleviate energy crisis and reduce carbon emissions. Due to clean energy resources e.g., photovoltaic, wind power and low-grade heat distribute in region which results to be difficult to integrate, this paper provide improved two-steps k-means arithmetic for cluster partition of Regional Integrated Energy System for utilizing distributed energy. Firstly, the weight index is defined for direction of cluster partition. Secondly, according to the resource endowment and located position of distributed energy, the initial energy cluster partition and cluster center considering weight coefficient is determined. Finally, load is divided into the cluster according to the distance of source load and matching coefficient. The practical case indicate that the method proposed can effectively realize cluster partition and integration of distributed energy.

HATS: HetTask Scheduling

To handle task execution, modern supercomputers employ thousands (or millions) of processors. In such supercomputers, task scheduling has a meaningful impression on system performance. To improve efficiency, task scheduling algorithms aim to decrease the volume of communication and the number of message exchanges. These efforts, however, result in other bottlenecks, such as high-link congestion. In addition, the heterogeneity of processors and networks is another major challenge for schedulers. This paper presents a new algorithm for scheduling called Heterogeneity-Aware Task Scheduling (HATS). The proposed algorithm adopts an updated multi-level hyper-graph partitioning approach. It describes a new method of aggregation in the coarsening step that helps to accurately coarsen the hyper-graph of the task model. The Raccoon Optimization algorithm is then used in the initial partitioning phase, and in the un-coarsening phase, a novel refinement procedure optimises the initial partitions. The experiments on this approach showed that, compared to the other well-known algorithms, the proposed method offers better schedules with lower communication volume and imbalance ratio in a shorter time.

Cluster Partitions Optimization Using the Latent Class Analysis Technique

The paper explores the formation of a cluster partition using a system of bundles of vectors. Each object is represented by a vector with coordinates corresponding to its characteristic factors. The study focuses on the principles of changes in the total density of these bundles when objects move from one cluster to another. Based on the findings, an algorithm is proposed to reduce the total intracluster variance of the initial cluster partition.
 The proposed algorithm is based on a modern technique for analyzing latent classes. This technique emphasizes that the forming factors of objects within each cluster should be highly correlated. Since the number of forming factors is arbitrary, the algorithm replaces this requirement with the maximum possible proximity of object factor vectors to the average vector within each cluster. The degree of this closeness is referred to as the tightness of the cluster vector bundle.
 The paper also introduces a new method for quantifying clusters in the improvable clustering constructed using the proposed algorithm. A practical example of the algorithm's application to process medical data is presented. The study discusses the reasons for the dependence of the algorithm's outcome on the choice of initial cluster partition.

Heterogenous structure and formation mechanism of white and brown etching layers in bainitic rail steel

The failures of conventional pearlitic rail steels are strongly affected by the formation of hard and brittle white and brown etching layers (WEL and BEL, respectively) on the rail raceway during service. Bainitic rail steels with an excellent strength–toughness balance are promising candidates for next generation rail steels. It is generally difficult to generate hard and brittle WEL/BEL in bainitic rails because of the low carbon content. This study reports, for the first time, the formation of a unique multilayer heterostructured WEL/BEL in a field-tested bainitic rail. The heterogenous structures of WEL/BEL were characterized using transmission electron microcopy, focused ion beam, energy-dispersive X-ray spectroscopy, electron energy loss spectroscopy, electron probe microanalysis, atom probe tomography, and nano-indentation techniques. Results demonstrate that WEL is mainly composed of fine-grained martensite or ferrite and retained austenite, whereas BEL is a composite structure containing nanocrystalline martensite or ferrite, retained austenite, cementite, and oxide or O-rich particles. The oxide or O-rich particles are primarily formed within BEL through local oxidation, resulting in secondary partitioning of the alloying elements between the oxidation product and its surrounding matrix. BEL is hard but brittle due to the formation of brittle twinned martensite, cementite, and oxide particles, whereas the adjacent WEL is relatively soft and ductile. Based on these findings, the heterostructured WEL/BEL formation mechanism is preliminarily discussed in terms of inhomogeneous grain refinement, localized oxidation, chemical alterations (including both chemical segregation in the initial microstructure and secondary partitioning of the alloying elements during local oxidation), and phase transformation.

Jeffreys heat conduction in coupled semispaces subjected to interfacial heating

A Jeffreys heat conduction problem for coupled semispaces subjected to the action of an interfacial heat source was defined. An analytical solution of the problem was derived for a polynomial specific power of the heat source using the Laplace transform approach. The asymptotic and parametric analysis was performed for different ratios of thermal conductivities K1,2, thermal diffusivities k1,2, thermal relaxation times τ1,2 and coefficients α1,2 indicating the relative contribution of Fourier heat conduction. It was found that Jeffreys heat conduction results in continuous variation of the contact temperature, whilst its particular case — Cattaneo heat conduction — is accompanied by a step change of the contact temperature at the initial time. Another finding is that the initial heat partition occurs due to the ratio of K1α1/k1 and K2α2/k2 under Jeffreys heat conduction and due to the different ratio of K1/k1τ1 and K2/k2τ2 under Cattaneo heat conduction. The solution applicability was demonstrated on the simulation example of ultrashort laser pulse welding. The type of heat conduction was revealed to have qualitative and quantitative impacts on the contact temperature and heat fluxes.

Balanced Domain Partitioning for Software Defined Networks

Software Defined Networking (SDN) has been widely adopted to enable the network programmability by decoupling the controller logic from the data plane. Domain partitioning is essential for SDN-enabled communication networks to partition the oversized network topology into controllable and administrative domains. Existing domain partitioning algorithms failed to take the connectivity, balance constraint, compactness, and traffic cohesion into consideration simultaneously which are required by SDN communications. This paper formulates the domain partitioning problem and proposed a novel balanced network partitioning (BaDoP) algorithm, which can enhance the balance of the partitioned domains, and optimize the compactness and traffic cohesion with connectivity constraint always guaranteed. BaDoP employs Bubble algorithm to find initial domain partitioning, and improves the domain partitioning iteratively by moving border nodes among the partitioned domains guided by the negative cycles or shortest paths on an auxiliary graph which is an abstract view of the current domain partitioning. The simulation results indicates that our BaDoP algorithm outperforms the previous algorithms and can achieve significant improvement for the balance of the partitioned domains.

Customer Segmentation Using the Integration of the Recency Frequency Monetary Model and the K-Means Cluster Algorithm

Purpose: This research aims to do customer segmentation in retail companies by implementing the Recency Frequency Monetary (RFM) K-Means cluster model and algorithm optimized by the Elbow method.Methods: This study uses several methods. The RFM model method was chosen to segment customers because it is one of the optimal methods for segmenting customers. The K-Means cluster algorithm method was chosen because it is easy to interpret, implement, fast in convergence, and adapt, but lacks sensitivity to the initial partitioning of the number of clusters. To help classify each category of customers and know the level of loyalty, they use a combination of the RFM model and the K-Means method. The Elbow method is used to improve the performance of the K-Means algorithm by correcting the weakness of the K-Means algorithm, which helps to choose the optimal k value to be used when clustering.Result: This research produces customer segmentation 3 clusters with a Sum of Square Error (SSE) value of 25,829.39 and a Callinski-Harabaz Index (CHI) value of 36,625.89. The SSE and CHI values are the largest ones, so they are the optimal cluster values.Novelty: The application of the integrated RFM model and the K-Means cluster algorithm optimized by the Elbow method can be used as a method for customer segmentation.

Urban Road Network Partitioning Based on Bi-Modal Traffic Flows With Multiobjective Optimization

The recent extension of a macroscopic fundamental diagram (MFD) into a bi-modal MFD (or 3D-MFD) provides the relationship among the total network circulating flows and the accumulations of private vehicles and public buses. 3D-MFD reveals the contribution of large occupancy vehicles such as buses in improving urban transportation efficiency. A lot of bi-modal traffic management techniques are introduced based on 3D-MFD to improve the urban traffic efficiency without using detailed origin-destination (OD) information. However, similar to MFD, 3D-MFD is also highly affected by the heterogeneity of a road network. In order to form 3D-MFDs with low scatter to be utilized for further bi-modal traffic management, this paper proposes a partition method to cluster road links into several homogeneous regions for a bi-modal urban network. It is comprised of three layers named as initial partition, merging, and boundary adjusting. At the initial partition layer, Seeded Region Growing (SRG) and the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) are integrated to obtain a number of subregions. A modified Genetic Algorithm (GA) is developed to merge the subregions into larger regions at the merging layer. Then, boundary adjusting is performed by changing the region to which a boundary is clustered to optimize the result. Multi-sensor data collected from Shenzhen in China are utilized to verify the effectiveness of the proposed partition method.

A Fuzzy Entropy-Based Thematic Classification Method Aimed at Improving the Reliability of Thematic Maps in GIS Environments

Thematic maps of spatial data are constructed by using standard thematic classification methods that do not allow management of the uncertainty of classification and, consequently, evaluation of the reliability of the resulting thematic map. We propose a novel fuzzy-based thematic classification method applied to construct thematic maps in Geographical Information Systems. An initial fuzzy partition of the domain of the features of the spatial dataset is constructed using triangular fuzzy numbers; our method finds an optimal fuzzy partition evaluating the fuzziness of the fuzzy sets by using a fuzzy entropy measure. An assessment of the reliability of the final thematic map is performed according to the fuzziness of the fuzzy sets. We implement our method on a GIS framework, testing it on various vector and image spatial datasets. The results of these tests confirm that our thematic classification method provide thematic maps with a higher reliability with respect to that obtained through fuzzy partitions constructed by expert users.

A Cluster-Based Partition Method of Remote Sensing Data for Efficient Distributed Image Processing

Data stream partitioning is a fundamental and important mechanism for distributed systems. However, use of an inappropriate partition scheme may generate a data skew problem, which can influence the execution efficiency of many application tasks. Processing of skewed partitions usually takes a longer time, need more computational resources to complete the task and can even become a performance bottleneck. To solve such data skew issues, this paper proposes a novel partition method to divide on demand the image tiles uniformly into partitions. The partitioning problem is then transformed into a uniform and compact clustering problem whereby the image tiles are regarded as image pixels without spectrum and texture information. First, the equal area conversion principle was used to select the seed points of the partitions and then the image tiles were aggregated in an image layout, thus achieving an initial partition scheme. Second, the image tiles of the initial partition were finely adjusted in the vertical and horizontal directions in separate steps to achieve a uniform distribution among the partitions. Two traditional partition methods were adopted to evaluate the efficiency of the proposed method in terms of the image segmentation testing, data shuffle testing, and image clipping testing. The results demonstrated that the proposed partition method solved the data skew problem observed in the hash partition method. In addition, this method is designed specifically for processing of image tiles and makes the related processing operations for large-scale images faster and more efficient.

Efficient game theoretic approach to dynamic graph partitioning

As a building block in many graph-based applications, graph partitioning aims to divide a graph into smaller parts of roughly equal size, and meanwhile, minimize the number of cutting edges. Existing solutions for graph partitioning are mainly designed for static graphs and are not appropriate for many dynamic graphs in real-world scenarios, including social networks, knowledge graphs, and web graphs. Although there is an incremental method, called IncKGGGP, proposed to efficiently deal with dynamic graphs, it can only be deployed on top of a specific batch partitioning algorithm, called KGGGP, which inherently impairs the final partitioning quality.To alleviate these issues, in this paper, we propose a novel Edge-Cut Partitioning approach based on Game theory for dynamic graphs (ECPG). Generally, ECPG is equipped with the following two nice properties. (1) High effectiveness. It can cope with dynamic graphs on top of any initial partitioning result, and then, achieve higher-quality results by choosing more effective static algorithms. (2) High efficiency. By exerting the advantage of game theory, ECPG can not only assign updated vertices into desirable partitions efficiently based on a low time complexity function but also can reduce redundant computations by reusing existing partitioning results. We also prove that there exists a Nash equilibrium in ECPG. From experimental results over several real-world graphs, it demonstrates that ECPG significantly outperforms the existing algorithms by up to one order of magnitude with comparable partitioning quality.

A Novel Data-Driven Approach to Autonomous Fuzzy Clustering

In this article, a new data-driven autonomous fuzzy clustering (AFC) algorithm is proposed for static data clustering. Employing a Gaussian-type membership function, AFC first uses all the data samples as microcluster medoids to assign memberships to each other and obtains the membership matrix. Based on this, AFC chooses these data samples that represent local models of data distribution as cluster medoids for initial partition. It then continues to optimize the cluster medoids iteratively to obtain a locally optimal partition as the algorithm output. Moreover, an online extension is introduced to AFC enabling the algorithm to cluster streaming data chunk-by-chunk in a “one pass” manner. Numerical examples based on a variety of benchmark problems demonstrate the efficacy of the AFC algorithm in both offline and online application scenarios, proving the effectiveness and validity of the proposed concept and general principles.

Power Balance Partition Control Based on Topology Characteristics of Multi-Source Energy Storage Nodes

Aiming at the power balance control of multi-source energy storage grid in the case of a high proportion of new energy grid connection. In this article, a power grid dynamic partition method based on the Markov energy field principle and a priori knowledge model is proposed. Combined with the coordinated dispatching of power grid source-load, a two-layer power balance partition control model based on the topological characteristics of multi-source energy storage nodes is established. First, in the upper-layer model, the energy homogenization method of multi-source energy storage nodes is studied, and the Markov energy field model of power grid node partition based on energy interaction constraints between nodes is established to partition the power grid initially. Combined with the prior model of node dynamic partition, the initial partition is dynamically optimized to realize the dynamic partition of the multi-source energy storage grid. Then, in the lower-layer model, the source-load coordinated dispatching model in the power grid partition area is established to realize the dynamic partition control of the power grid. Finally, based on the real operation data of a northeast power grid and IEEE39 node system, a dynamic partition power control simulation model of a multi-source energy storage power grid is established. The simulation results and analysis show that the dynamic partition power control strategy proposed in this article can effectively improve the regulation ability and economy of the power grid.

Automatic Dynamic Range Adjustment for Pedestrian Detection in Thermal (Infrared) Surveillance Videos.

This paper presents a novel candidate generation algorithm for pedestrian detection in infrared surveillance videos. The proposed method uses a combination of histogram specification and iterative histogram partitioning to progressively adjust the dynamic range and efficiently suppress the background of each video frame. This pairing eliminates the general-purpose nature associated with histogram partitioning where chosen thresholds, although reasonable, are usually not suitable for specific purposes. Moreover, as the initial threshold value chosen by histogram partitioning is sensitive to the shape of the histogram, specifying a uniformly distributed histogram before initial partitioning provides a stable histogram shape. This ensures that pedestrians are present in the image at the convergence point of the algorithm. The performance of the method is tested using four publicly available thermal datasets. Experiments were performed with images from four publicly available databases. The results show the improvement of the proposed method over thresholding with minimum-cross entropy, the robustness across images acquired under different conditions, and the comparable results with other methods in the literature.

Optimal selection of touristic packages based on user preferences during sports mega-events

Sport mega-events, such as the Soccer World Cup or Olympic Games, attract many visitors from all over the world. Most of these visitors are also interested in, besides attending the sports events, visiting the host nation and the neighboring countries. In this paper, we focus on the upcoming FIFA World Cup Qatar 2022. As per the schedule of the tournament, a national team can play 7 matches at most. Therefore, a supporter will have six short breaks (of three to five days) between consecutive matches in addition to two longer ones, immediately before and after the tournament, during which they can plan some touristic trips. We study the problem faced by a touristic trip provider who wants to offer a set of touristic packages, chosen among a very large set of options, devoted to World-Cup related tourists. The number of packages offered must be limited due to organizational reasons and the necessity to guarantee a high participation in each trip. In this study, a set of user profiles is considered. It represents different categories of tourists, characterized by different preferences and budgets. Each user is supposed to pick the packages that maximize their satisfaction, considering their budget and time restraints. The goal of the company is to choose the set of packages to be offered that would maximize the average users satisfaction. To address this NP-Hard combinatorial optimization problem we provide a mathematical formulation and a matheuristic, named Consensus-Based Kernel Search (CKS), wherein an alternative rule is used to create the initial Kernel and partition variables in buckets. Computational results evidence the excellent performance of CKS and prove that the newly introduced algorithm systematically outperforms the classical Kernel Search.

Ti-6Al-4V alloy strengthening via instantaneous phase transformation induced by electropulsing

High-strength titanium alloys are necessary and essential in aviation and aerospace fields. Attempt to enhance the strength of the most widely used Ti-6Al-4V alloy is significant. However, it is hard to obtain high strength (>1300 MPa) combining with adequate ductility (elongation>12%) for this alloy. In this study, Ti-6Al-4V alloy with high yield strength (1280 MPa), excellent ultimate tensile strength (1358 MPa) and proper elongation (13.7%) was successfully prepared by an innovative strategy based on electropulsing. The strong electron flow of electropulsing induced rapid temperature rise and yielded instantaneous phase transformation, which limited alloying elements diffusing and parent β size growing, and followed by different phase transformations in Al or V solute enriched regions in one parent β grain during the cooling process. The particular microstructure evolution resulted in efficient α′ lath refinement. Then, the strength can be greatly enhanced by the decomposition of fine α' after subsequent aging. Meanwhile, the V and Al generally kept the initial microstructural partition and resulted in a novel bi-lamellar microstructure, which led to proper ductility. The results and analysis provide a novel method and new idea to improve and control the mechanical properties of Ti-6Al-4V and other titanium alloys.