Partitioning Technique Research Articles

Scalable Data Partitioning Techniques for Distributed Data Processing in Cloud Environments: A Review

Scalable Data Partitioning Techniques for Distributed Data Processing in Cloud Environments: A Review

A Comparative Study of the Performance of Machine Learning Models on a Tax Dataset of Yemen to Detect Levels of Tax Evasion

The performance of a classification model in machine learning is affected by many factors, such as the type of machine learning technology used. Accuracy varies from method to method. This paper presents a comparison between the performance of different models in terms of the machine learning technique used (e.g. KNN, NB, SVM, DT, RF, MLP). Based on the data provided by the Tax Authority of Yemen, which is related to the commercial and industrial profits tax, which consists of 760 attributes, after the preprocessing of data. The dataset partition technique used k-fold validation. The paper shows that the e Naïve Bayes (NB) classifier gave the highest result in accuracy and other measures. Then KNN, SVM, and RF gave the same results in accuracy 99.87%, but in SVM, KNN the results were also the same in the rest measures, while in RF models the rest measures were 97.91%,99.95%, and 98.91% in Recall, Precision and F-score in order. MLP gave 98.42 in accuracy with 66.62%, 64.21%, and 64.40 in the recall, Precision, and F-score, then DT gave 97.76% in accuracy with 57.006% ,99.24% and 72.41% in the recall, precision, and F-score.

Practical NTRU Signcryption in the Standard Model.

Based on the NTRU trapdoor used in NIST's Falcon, a signcryption scheme following the sign-then-encrypt paradigm is constructed. The existing partitioning technique based on Waters hash over the lattice can not complete the security reduction in the standard model for the signature part due to the "partiality" of the pre-image generated with the NTRU trapdoor. To address this, a variant of Waters hash over small integers is proposed and, the probability of the successful reduction is analyzed. The resulting signcryption achieves existential unforgeability under the adaptive chosen-message attacks. By utilizing the uniqueness of the secret and the noise in an NTRU instance, the tag used in encryption is eliminated. Furthermore, a method to construct tamper-sensitive lattice public key encryption is proposed. This approach implants the ciphertext-sensitive information into the lattice public key encryption and binds it to the encrypted information. The malleability to the public key ciphertext triggers the change of the message-signature pair so that the IND-CCA2 security of the entire ciphertext can be guaranteed by the signature for the message. Thanks to the rational design and the efficiency of the NTRU trapdoor, the computational overhead of the proposed scheme is reduced significantly compared to the existing lattice-based signcryption scheme, reaching orders of magnitude improvement in efficiency. The experiment shows that the proposed scheme is efficient.

Categorization of Indian residential consumers electrical energy consumption pattern using clustering and classification techniques

Data Mining techniques are widely applied in power sector to analyze the electricity consumption pattern(ECP) of consumers for implementing effective demand response management. In this study, the ECP of LV consumers of Karur city is used for evaluation. The dataset consists of five years ECP. In this study, K-means clustering algorithm is used to categorize the data into binary and three classes. The results of the K-means clustering technique is used for developing eight classifier models. The hyperparameters of the classifier models are tuned using grid search optimization technique. The classification process applies hold-out partitioning and stratified 10-fold cross-validation(CV) techniques to evaluate the classifier’s efficiency in predicting a consumer’s ECP. The experimental result shows that the K-means clustering has minimal overlapping of 0.48% and 1.98% for binary and three-class clustering processes. Hence, the K-means algorithm results are used to build this study’s classification model. In the classification process, the experimental results of LR and SVM algorithms have better accuracy in binary and three class categories respectively, for the hold-out partitioning method. Meanwhile, the SVM also has better generalization ability and is validated using stratified 10-fold CV, achieving an accuracy of 99.09% and 98.19% for binary and three classes respectively, in the classification of consumer data.

LAG-based schedulability analysis for preemptive global EDF scheduling with dynamic cache allocation

In recent years, the maturation of modern multicore processor technology and its increasing adoption in critical industrial domains have posed significant challenges for real-time systems, primarily due to contention for shared cache resources and the resulting uncertainty. To address this issue, contemporary processors employ cache partitioning techniques, enhancing temporal predictability by isolating cache access among processor cores. However, this isolation technique may lead to real-time tasks missing their deadlines due to an insufficient number of cache partitions. Consequently, this paper investigates the schedulability of preemptive global Earliest Deadline First (EDF) real-time scheduling algorithms that support dynamic cache allocation. We propose an innovative LAG-based schedulability analysis method for these algorithms and present a utilization-based schedulability condition that reduces analysis time complexity while improving analysis accuracy. Building upon this foundation, and incorporating task characteristics into our analysis, this paper further introduces an optimization technique aimed at minimizing the pessimism inherent in the schedulability test. Lastly, the performance and efficiency of the proposed schedulability determination method are validated through simulation experiments with randomly generated tasks.

Cache-Efficient Top-k Aggregation over High Cardinality Large Datasets

Top-k aggregation queries are widely used in data analytics for summarizing and identifying important groups from large amounts of data. These queries are usually processed by first computing exact aggregates for all groups and then selecting the groups with the top-k aggregate values. However, such an approach can be inefficient for high-cardinality large datasets where intermediate results may not fit within the local cache of multi-core processors leading to excessive data movement. To address this problem, we have developed Zippy, a new cache-conscious aggregation framework that leverages the skew in the data distribution to minimize data movements. This is achieved by designing cache-resident data structures and an adaptive multi-pass algorithm that quickly identifies candidate groups during processing, and performs exact aggregations for these groups. The non-candidate groups are pruned cheaply using efficient hashing and partitioning techniques without performing exact aggregations. We develop techniques to improve robustness over adversarial data distributions and have optimized the framework to reuse computations incrementally for rolling (or paginated) top-k aggregate queries. Our extensive evaluation using both real-world and synthetic datasets demonstrate that Zippy can achieve a median speed-up of more than 3× for monotonic aggregation functions across typical ranges of k values (e.g., 1 to 100) and 1.4× for non-monotonic functions when compared with state-of-the-art cache-conscious aggregation techniques.

Multi-Objective Genetic Algorithm-Based Technique for Achieving Low-Power VLSI Circuit Partition

Minimizing the power consumption of electronic systems is one of the most critical concerns in the design of integrated circuits for very large-scale integration (VLSI). Despite the reality that VLSI design is known for its compact size, low power, low price, excellent dependability, and high functionality, the design stage remains difficult to improve in terms of time and power. Several optimization algorithms have been designed to tackle the present issues in VLSI design. This study discusses a bi-objective optimization technique for circuit partitioning based on a genetic algorithm. The motivation for the proposed research is derived from the basic concept that, if some portions of a circuit's system are deactivated during the processor's idle time, the circuit's power consumption is automatically reduced. To reduce the overall system's power consumption, maximization of sleep time and minimization of net cuts are required. To achieve these, an effective fitness function has been constructed in such a way that the balance criteria are also maintained. The approach has been tested on a set of net lists from the ISPD'98 benchmark suite, each containing 10 to 30 nodes. The experimental results are compared with two existing methods that clearly indicate the acceptability of the suggested method. The suggested strategy achieves an average reduction of 24.69% and 31.46% for net cut, whereas average extensions of 15.20% and 12.31% are observed in sleep time when compared with two existing methods. The proposed method also achieves an average power efficiency of 14.98% and 12.09% with respect to these two state-of-the-art methods.

Optimal locating and sizing of charging stations for large‐scale areas based on GIS data and grid partitioning

AbstractThis research proposes an optimisation model for identifying the optimal locations and sizes of public charging stations for large‐scale areas based on geographic information system (GIS) data and grid partitioning. A grid partitioning technique is utilised to generate subareas and investigate the demand for electric vehicle (EV) chargers calculated based on the number of EVs in each subarea. The optimal locations are formulated as a maximum covering location problem and solved by integer programming to ensure that all vehicle access subareas, according to the GIS data, are covered by charging stations. The sizes of charging stations are formulated as a knapsack problem and solved by the EV‐charge point ratio to guarantee each subarea has the appropriate number of chargers. The Voronoi diagram is integrated to estimate vehicle density and improve visualisation. The model is extended to include potential power loss reduction by adjusting the distribution of chargers in relation to the proximity to electrical substations. The developed methodology has been implemented by the whole country area of Thailand, which covers an area of approximately 510,000 km2. The study results are represented in a map blueprint that is very useful in strategic planning to support the so‐called EV30@30 government policy's ambitious national target of 2,050,000 units of EVs in 2030.

A machine learning framework for remaining useful lifetime prediction of li-ion batteries using diverse neural networks

Accurate prediction of the remaining useful life (RUL) of lithium-ion batteries (LIBs) is pivotal for enhancing their operational efficiency and safety in diverse applications. Beyond operational advantages, precise RUL predictions can also expedite advancements in cell design and fast-charging methodologies, thereby reducing cycle testing durations. Despite artificial neural networks (ANNs) showing promise in this domain, determining the best-fit architecture across varied datasets and optimization approaches remains challenging. This study introduces a machine learning framework for systematically evaluating multiple ANN architectures. Using only 30% of a training dataset derived from 124 LIBs subjected to various charging regimes, an extensive evaluation is conducted across 7 ANN architectures. Each architecture is optimized in terms of hyperparameters using this framework, a process that spans 145 days on an NVIDIA GeForce RTX 4090 GPU. By optimizing each model to its best configuration, a fair and standardized basis for comparing their RUL predictions is established. The research also examines the impact of different cycling windows on predictive accuracy. Using a stratified partitioning technique underscores the significance of consistent dataset representation across subsets. Significantly, using only the features derived from individual charge–discharge cycles, our top-performing model, based on data from just 40 cycles, achieves a mean absolute percentage error of 10.7%.

An efficient 3D IC partitioning approach using satin bowerbird optimization for reduced TSV count and improved heat dissipation

Net list partitioning achieves paramount importance in the physical architecture design step of Three Dimensional (3D) Very Large Scale Integrated (VLSI) circuits. The performance of all subsequent steps like floor layout, placement, pin assignment, and routing in the physical architecture design of the VLSI circuit are heavily affected by the outcomes of partitioning steps. Wire length between gates is reduced by 3D Integrated Circuit (IC) due to compact footprint as well as vertical inter connections among dies. This reduced wire length in turns responsible for less energy consumption and reduced connection delays. In this paper, Satin Bowerbird Optimization (SBO) is applied to solve 3D IC partitioning challenges. The suggested 3D IC partitioning technique is aimed at reducing the number of Through Silicon Vias (TSV) and measurements have also been taken to reduce heat dissipation. The presence of a significant amount of TSVs expands the chip area. 3D ICs have considerably greater densities of power because of advances in the technology and a vastly increased variety of components with high frequency. The heat generated by the used power has an impact on the performance and dependability of the chip. In this experiment, SBO is used as a multi-objective optimization method to achieve two objectives simultaneously - TSV count minimization and heat dissipation reduction. The performance of the suggested SBO - based approach is assessed utilizing the Giga Scale Research Center (GSRC) benchmark circuits. The applicability of the suggested technique is judged by comparing the outcomes with thermal-aware multilevel hyper-graph partitioning method. The findings of the comparison reveal that the suggested technique outperforms the thermal-aware multilevel hyper-graph partitioning approach by lowering the number of TSV count, maximum area requirement, and power density by 16.99%, 9.94% and 29.10% respectively. The performance of the experiment is also compared with existing Simulated Annealing and Tabu search based techniques. In both cases, the proposed method achieves better result by lowering TSV by 26.03% and 27.31%, and reducing area by 6.59% and 6.59%, respectively.

Large‐scale homo‐ and heterogeneous parallel paradigm design based on CFD application PHengLEI

SummaryThe development of computational fluid dynamics (CFD) highly depends on high‐performance computers. Computer hardware has evolved rapidly, yet scalable CFD parallel software remains scarce. In this article, we design a highly scalable CFD parallel paradigm for both homogeneous and heterogeneous supercomputers. The paradigm achieves the separation of communication and computation and automatically adapts to various solvers and hardware environments, thus reducing programming difficulties and increasing automatic parallelization. Meanwhile, the number of communications is greatly reduced and the scalability of the program is improved through implementing centralized communication and two‐level partitioning techniques. Complex flow problems for real aircraft were then computed on different hardware platforms with a grid size of ten billion. The homogeneous computer hardware includes Intel Xeon Gold 6258R and Phytium 2000+ processors, and the heterogeneous computer platforms include NVIDIA Tesla V100 and SW26010 processors. High parallel efficiency was obtained on all computer platforms, verifying that the paradigm has good automatic parallelization, scalability, and stability. The paradigm in this article has an important reference value for CFD massively parallel computing and can promote the development and application of CFD technology.

A Novel and Efficient Spatio-Temporal Clustering Algorithm to Discover Patterns in Indian Earthquake Data

The seismic activity in India, including the Himalayas, the North-East region, and the Andaman-Nicobar Islands, is prominently displayed on the seismic map. It is important to analyze the specific characteristics of seismic events on the Indian sub-continent. This research paper introduces a new algorithm for data analysis using a partitioning technique that mines clustering patterns. The algorithm generates clusters of spatial and spatio-temporal data by distributing the data into spatial bins or buckets, identifying neighboring bins, and minimizing distance calculations. Furthermore, the selection of centroids is based on the density of data in the spatial region, utilizing a random selection method. It requires minimal parameter settings, enhancing its efficiency and practicality for analyzing large-scale spatio-temporal datasets. We validate the algorithm's efficacy through experiments conducted on the Indian earthquake spatio-temporal dataset, demonstrating its ability to detect spatio-temporal patterns and identify earthquake-prone regions.The conducted experiments reveal a correlation between the frequency of earthquake events and the number of clusters formed, indicating that regions with a higher occurrence of earthquakes exhibit a greater clustering tendency, signifying their susceptibility to seismic activity. The results imply promising clustering quality, with Silhouette index in the range of 0.88 to 0.93.

Coarse-grained multicomputer parallel algorithm using the four-splitting technique for the minimum cost parenthesizing problem

Dynamic programming is a technique widely used to solve several combinatory optimization problems. A well-known example is the minimum cost parenthesizing problem (MPP), which is usually used to represent a class of non-serial polyadic dynamic-programming problems. These problems are characterized by a strong dependency between subproblems. This paper outlines a coarse-grained multicomputer parallel solution using the four-splitting technique to solve the MPP. It is a partitioning technique consisting of subdividing the dependency graph into subgraphs (or blocks) of variable size and splitting large-size blocks into four subblocks to avoid communication overhead caused by a similar partitioning technique in the literature. Our solution consists in evaluating a block by computing and communicating each subblock of this block to reduce the latency time of processors which accounts for most of the global communication time. It requires O(n^3/p) execution time with O(k * \sqrt{p}) communication rounds. n is the input data size, p is the number of processors, and k is the number of times the size of blocks is subdivided.

PP-Bridge: Establishing a Bridge between the Prefetching and Cache Partitioning

— Modern computer processors are equipped with multiple cores, each boasting its own dedicated cache memory, while collectively sharing a generously sized Last Level Cache (LLC). To ensure equitable utilization of the LLC space and bolster system security, partitioning techniques have been introduced to allocate the shared LLC space among the applications running on different cores. This partition dynamically adapts to the requirements of these applications. Prefetching plays a vital role in enhancing cache performance by proactively loading data into the cache before it get requested explicitly by a core. Each core employs prefetch engines to decide which data blocks to fetch preemptively. However, a haphazard prefetcher may bring in more data blocks than necessary, leading to cache pollution and a subsequent degradation in system performance. To maximize the benefits of prefetching, it is essential to keep cache pollution to a minimum. Intriguingly, our research has uncovered that when existing prefetching techniques are combined with partitioning methods, they tend to exacerbate cache pollution within the LLC, resulting in a noticeable decline in system performance. In this paper, we present a novel approach aimed at mitigating cache pollution when combining prefetching with partitioning techniques.

On Degree-Based Topological Indices of Kagome Graphene, and Carbon Kagome Nanotubes and Nanotori

Approaches based on graph descriptors have been used in cheminformatics and bioinformatics for molecular property prediction. In this article, we apply partition technique and simple counting schemes to describe the structure of a two-dimensional (2D) kagome graphene and then derive the closed form of various degree-based topological indices (graph descriptor) of 2D kagome graphene, and its tubular and toroidal forms.

An efficient and scalable SPARQL query processing framework for big data using MapReduce and hybrid optimum load balancing

The increasing RDF (Resource Description Framework) data volume requires a Hadoop platform for processing queries over large datasets. In this work, SPARQL (Simple Protocol and Rdf Query Language) queries are evaluated with Hadoop based on the objective of minimizing the number of joins through data partitioning for performing map/reduce jobs. The query evaluation time and the number of cross node joins are minimized with the proposed partitioning techniques. Extended vertical partitioning is proposed for distributed data stores based on objects’ explicit information for splitting predicates. For accessing the RDF data, hybrid monarch butterfly with beetle swarm load balancing optimization with Map-reduce (Hybrid Optimum Load Balancing) is applied. The proposed SPARQL query processing is evaluated over large RDF datasets. The proposed approach’s evaluation results are analyzed with the existing approaches, indicating the proposed framework’s efficiency. By using the proposed approach, an accuracy of 97 % is obtained.

SPAW-SMOTE: Space Partitioning Adaptive Weighted Synthetic Minority Oversampling Technique For Imbalanced Data Set Learning

Abstract The problem of data imbalance is common in reality, which greatly affects the performance of classifiers. Most of the solutions are to balance the data set by generating new minority class samples, which are faced with the problems of selecting the appropriate area for generating samples, fuzzy classification boundary and uneven distribution of samples. To solve these problems, we propose a novel oversampling algorithm named space partitioning adaptive weighted synthetic minority oversampling technique (SPAW-SMOTE). We first divide the data space into boundary space and non-boundary space based on spatial partitioning techniques. The number of samples to be generated is assigned to different spaces by the designed adaptive weighting algorithm, which is used to solve the problems of uneven distribution of samples and easy to blur the classification boundary. Finally, we also endeavor to develop a new generation algorithm to reduce the probability of overlapping samples generated when synthesizing new samples and to ensure the diversity of new samples. Experimental results on 18 real-world data sets show that the average performance (G-mean, F1-measure and Area Under Curve) of SPAW-SMOTE is significantly better than other existing oversampling techniques.

Secretary Problems with Biased Evaluations Using Partial Ordinal Information

The k-secretary problem deals with online selection of at most k numerically scored applicants, where selection decisions are immediate upon their arrival and irrevocable, with the goal of maximizing total score. There is, however, wide prevalence of bias in evaluations of applicants from different demographic groups (e.g., gender, age, race), and the assumption of an algorithm observing their true score is unreasonable in practice. In this work, we propose the poset secretary problem, where selection decisions must be made by observing a partial order over the applicants. This partial order is constructed to account for uncertainty and biases in applicant scores. We assume that each applicant has a fixed score, which is not visible to the algorithm and is consistent with the partial order. Using a random partitioning technique from the matroid secretary literature, we provide order-optimal competitive algorithms for the poset secretary problem and provide matching lower bounds. Further, we develop the theory of thresholding in posets to provide a tight, adaptive-thresholding algorithm under regimes where k grows quickly enough, thus matching the adaptiveness to k shown for the classical k-secretary problem. We then study a special case, in which applicants belong to g disjoint demographic groups and the bias is group specific. We provide competitive algorithms for adversarial and stochastic variants of this special case, including a framework, Gap, for parallelizing any vanilla k-secretary algorithm to the group setting. Finally, we perform a case study on real-world data to demonstrate the responsiveness of our algorithms to data and their impact on selection rates. This paper was accepted by Hamid Nazerzadeh, data science. Funding: This work was supported by the National Science Foundation [Grant 2112533]. S. Gupta’s research was supported in part by NSF CAREER Award 2239824. Supplemental Material: The data files are available at https://doi.org/10.1287/mnsc.2023.4926 .

Automated Testing of Indo Towing Information System with Katalon Studio

The development of web-based and mobile information systems that support various aspects of human work has become the main focus in the current technological era. The importance of automation testing in ensuring software quality is becoming increasingly profound. The quality of these systems is critical to supporting successful business operations. Automation testing using the STLC method and equivalence partitioning technique is applied to ensure the quality of the Indo Towing system. The testing tool used is Katalon Studio, which facilitates the implementation of automated testing with efficiency. Thus, this study emphasizes the importance of automation testing as a crucial step in software development. The results of the implementation of automation testing using Katalon Studio were successfully carried out according to the information system function without any bugs being found in the system. Tests carried out automatically using Katalon Studio can be concluded that the system is running well.

A CPU parallelized coupling framework for constrained flexible multibody system and granular matter based on contact-graph

This paper proposes a coupling model for simulating the interaction between particles and the flexible multibody system. The flexible rods are described using the Absolute Nodal Coordinate Formulation, and various types of contacts, including particle-particle, particle-rod, particle-rigid body contacts, and contacts within multibody objects, are considered. The contact detection is improved by combining space partitioning techniques and the bounding box method. A data structure based on a contact graph is introduced to store the contact relationships efficiently. The discrete element system and multibody system are coupled through serial time integration, with data exchanges at the program level. Numerical examples demonstrate the validity of the proposed model. Moreover, the model is applied to simulate the asteroid soil sampling with flexible brush sampler, showing good agreement with experimental data. The analysis of engineering examples provides insights for sampler design, and show the model's engineering value.