Partitioning Scheme Research Articles

A frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning

The construction of microgrid has fully promoted the large-scale access of distributed power and the rapid development of electric vehicles (EVs). And microgrid system faces operational issues such as strong random disturbances from distributed power sources and loads, as well as unexpected events during operation. These issues can lead to unstable microgrid frequency, excessive discharge of EVs, and increased control costs. To solve the issues, a frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning (EDRL) is proposed in this paper. First, consider the effect of the vehicle-to-grid (V2G) process on the minimum time required to fully charge an EV, and the impact of the output distribution of micro turbines (MTs) on the regulation cost, and based on the coupling relationship between generator terminal voltage regulation and system frequency control, a multimicrogrid comprehensive control model is constructed. Second, in order to deal with engineering tasks with deceptive rewards and sparse rewards such as integrated frequency control, evolutionary algorithms and deep reinforcement learning algorithms are combined to effectively assist the training process to jump out of local optimal solutions and approach optimal control strategies. Meanwhile, the algorithm is improved based on the novelty search and intelligent partition strategy, so that the strategy has better convergence characteristics and can reduce the cost of information transmission and computational complexity under the premise of ensuring the control effect. Furthermore, the state space, action space and reward function of the controller are defined. Finally, the simulation results show that the proposed controller has the ability of coordinated control, can effectively reduce the adjustment cost of MT unit and the unnecessary discharging of EVs while ensuring the frequency regulation requirements of each submicrogrid, which is far superior to PID control, fuzzy control, and traditional deep reinforcement learning control.

Hybrid model-data-driven dynamic VAR planning for wind-penetrated power system using spectral surrogate techniques

The increasing wind power penetration and proliferation of induction motor loads, of which dynamic impact cannot be revealed through steady-state analysis, bring challenges to the short-term voltage stability of modern power systems. This study proposes a hybrid model-data-driven approach for dynamic VAR source planning to enhance the short-term voltage stability of wind-penetrated power systems to reduce the computation burden of electro-mechanical transient models. Firstly, the theoretical background of Stochastic Spectral Embedding (SSE) is introduced. Then, a surrogate model for the electromechanical transient model is established using SSE following efficient expansion coefficient calculation and a designed partition strategy for the studied problem. Furthermore, a hybrid model-data-driven VAR deployment optimization model is established with 3 objectives, which is solved by a dual-population-based evolutionary algorithm (DPEA). The accuracy and effectiveness of the model are verified on a modified New England 39-bus system. Simulation results prove that the computational cost is reduced significantly in comparison with conventional model-based method without a compromise in accuracy and the proposed method is also more accurate than methods based on other surrogate models. The proposed SSE-based model can be applied to other power system analysis with electro-mechanical transient models to alleviate the computational cost.

Uncertainty propagation analysis for distributed hydrological forecasting using a neural network

AbstractThe last few decades have presented a significant increase in hydrological disasters, such as floods. In some countries, most of the environmental, socioeconomic, and biodiversity losses are caused by floods. Thus, flood forecasting is crucial to support an efficient disaster warning system. This work proposes a model for hydrological forecasting based on a neural network with a geographically aligned input named GeoNN. It employs weather radar data to obtain accumulated rainfall in each grid cell of the watershed and make 15‐ and 120‐min predictions of the outlet river level. An uncertainty propagation analysis was performed for GeoNN from a collection of test cases obtained by either using different schemes of the dataset partitioning or introducing different additive‐noise rates to the input data to provide a probability of flood occurrence and also an ensemble prediction. Both this probability and the ensemble were able to detect occurrences of river levels exceeding a given flood threshold.

Source apportionment of soil PTE in a northern industrial county using PMF model: Partitioning strategies and uncertainty analysis

Positive matrix factorization (PMF) has commonly been applied for source apportionment of potentially toxic elements (PTE) in agricultural soil, however, spatial heterogeneity of PTE significantly undermines the accuracy and reliability of PMF results. In this study, a representative industrial-agricultural hub in North China (Xuanhua district, Zhangjiakou City) was selected as the research subject, multiple partition processing (PP) strategies and uncertainty analyses were integrated to advance the PMF modeling and associated algorithm mechanisms were comparatively discussed. Specifically, we adopted three methods to split the research area into several subzones according to industrial density (PP-1), population density (PP-2), and the ecological risk index (PP-3) respectively, to rectify the spatial bias phenomenon of PTE concentrations and to achieve a more interpretable result. Our results indicated that the obvious enrichment of Cd, Pb, and Zn was found in the agricultural soil, with Hg and Cd accounted for 83.49% of the overall potential ecological risk. Combining proper PP with PMF can significantly improve the modelling accuracy. Uncertainty analysis showed that interval ratios of tracer species (Cd, Pb, Hg, and Zn) calculated by PP-3 were consistently lower than that of PP-1 and PP-2, indicating that PP-3 coupled PMF can afford the optimal modeling results. It suggested that natural sources, fertilizers and pesticides, atmosphere deposition, mining, and smelting were recognized as the major contributor for the soil PTE contamination. The contribution of anthropogenic activities, specifically fertilizers and pesticides, and atmosphere deposition, increased by 1.64% and 5.91% compared to PMF results. These findings demonstrate that integration of proper partitioning processing into PMF can effectively improve the accuracy of the model even at the case of soil PTE contamination with high heterogeneity, offering support to subsequently implement directional control strategies.

Speed vs. efficiency: A framework for high-frequency trading algorithms on FPGA using Zynq SoC platform

Software-based technical indicators have been widely used for the stock market forecasting, aiming to predict market direction. Even though many algorithms for the software based technical indicators are presented, there are almost no hardware implementations reported in the literature. In this paper, the hardware implementation is presented for three commonly used technical indicators: Moving Average Convergence/Divergence (MACD), Relative Strength Index (RSI), and Aroon. Latency evaluation is conducted for Bitcoin and Ethereum within a single-day timeframe, utilizing the Xilinx Zynq-7000 programmable SoC XC7Z020-CLG484-1 platform.Additionally, various hardware/software (HW/SW) partitioning strategies are explored to leverage the flexibility of software alongside the performance advantages of hardware via the Zynq SoC platform. The results show that the best performing technical indicator is MACD with a speedup of 30 times over its software only counterpart. Furthermore, a hybrid design integrating multiple technical indicators is proposed, pairing MACD with RSI due to their competitive throughput values, differing by only 0.38 microseconds. This hybrid approach capitalizes on the parallel processing capabilities of hardware, enabling multiple systems to operate simultaneously.

Partition refinement of WorldPop population spatial distribution data method: A case study of Zhuhai, China.

Currently, the core idea of the refined method of population spatial distribution is to establish a correlation between the population and auxiliary data at the administrative-unit level and, then, refine it to the grid unit. However, this method ignores the advantages of public population spatial distribution data. Given these problems, this study proposed a partition strategy using the natural break method at the grid-unit level, which adopts the population density to constrain the land class weight and redistributes the population under the dual constraints of land class and area weights. Accordingly, we used the dasymetric method to refine the population distribution data. The study established a partition model for public population spatial distribution data and auxiliary data at the grid-unit level and, then, refined it to smaller grid units. This method effectively utilizes the public population spatial distribution data and solves the problem of the dataset being not sufficiently accurate to describe small-scale regions and low resolutions. Taking the public WorldPop population spatial distribution dataset as an example, the results indicate that the proposed method has higher accuracy than other public datasets and can also describe the actual spatial distribution characteristics of the population accurately and intuitively. Simultaneously, this provides a new concept for research on population spatial distribution refinement methods.

Distributed Graph Processing in Cloud Computing: A Review of Large-Scale Graph Analytics

The rapid growth of graph data in various domains has propelled the need for efficient distributed graph processing techniques in cloud computing environments. This paper presents a comprehensive review of distributed graph processing for graph analytics of massive size in the context of cloud computing. The paper begins by highlighting the challenges associated with distributed graph processing, including load balancing, communication overhead, scalability, and partitioning strategies. It provides an overview of existing frameworks and tools specifically designed for distributed graph processing in cloud environments. Furthermore, the review encompasses various techniques and algorithms employed in distributed graph processing. The paper also reviews recent research advancements in optimizing distributed graph processing in cloud computing. To provide practical insights, the paper presents a comparative analysis of representative large-scale graph analytics applications implemented on different cloud computing platforms. Performance, scalability, and efficiency metrics are evaluated under varying workload sizes and graph characteristics. Overall, this comprehensive review paper serves as a highly prized asset for researchers and large-scale graph analytics professionals who are practitioners in the field. It provides a holistic understanding of the state-of-the-art distributed graph processing techniques in cloud computing and guides future research efforts towards more efficient and scalable graph processing in cloud environments.

Niche partitioning and individual specialisation in resources and space use of sympatric fur seals at their range margin

Ecological theory predicts niche partitioning between high-level predators living in sympatry as a mechanism to minimise the selective pressure of competition. Accordingly, male Australian fur seals Arctocephalus pusillus doriferus and New Zealand fur seals A. forsteri that live in sympatry should exhibit partitioning in their broad niches (in habitat and trophic dimensions) in order to coexist. However, at the northern end of their distributions in Australia, both are recolonising their historic range after a long absence due to over-exploitation, and their small population sizes suggest competition should be weak and may allow overlap in niche space. We found some niche overlap, yet clear partitioning in diet trophic level (δ15N values from vibrissae), spatial niche space (horizontal and vertical telemetry data) and circadian activity patterns (timing of dives) between males of each species, suggesting competition may remain an active driver of niche partitioning amongst individuals even in small, peripheral populations. Consistent with individual specialisation theory, broad niches of populations were associated with high levels of individual specialisation for both species, despite putative low competition. Specialists in isotopic space were not necessarily specialists in spatial niche space, further emphasising their diverse individual strategies for niche partitioning. Males of each species displayed distinct foraging modes, with Australian fur seals primarily benthic and New Zealand fur seals primarily epipelagic, though unexpectedly high individual specialisation for New Zealand fur seals might suggest marginal populations provide exceptions to the pattern generally observed amongst other fur seals.

An Adapted Ant Colony Optimization for Feature Selection

ABSTRACT As information technologies evolve, they generate vast and ever-expanding datasets. This wealth of high-dimensional data presents challenges, including increased computational demands and difficulties in extracting valuable insights. The aim of feature selection is to address this complexity by reducing data dimensions with minimal information loss. Our proposed feature selection approach, the Feature Selection via Ant Colony Optimization algorithm, employs heuristic distance directly in its probability function, instead of using its inverse. The algorithm bypasses the need for sub-attribute sets, running multiple iterations to create a frequency order list from the collected routes, which informs feature importance. The efficacy of this technique has been validated through comparative experiments with other methods from scientific literature. To ensure fairness, these experiments used identical datasets, data partitioning strategies, classifiers, and performance metrics. Initially, the algorithm was compared with fifteen different algorithms, and subsequently benchmarked against three selected methods. The impact of feature selection on classification performance was statistically verified through comparisons before and after the feature selection process. Convergence performance of the proposed method has also been evaluated. Our findings robustly support the efficacy of the introduced approach in managing complex, multidimensional data effectively.

Complex network theory and game theory-based partitioning decision-making of parallel restoration for resilient power grid

The ability of fast restoration reflects power system resilience and safety. This paper proposes a partitioning decision-making method for parallel restoration based on the label propagation algorithm and the cooperative game theory. The topological and physical characteristics of blackout systems are considered as the edge weight and the node importance simultaneously for formulating the partitioning model. By the label propagation algorithm, various labels representing different subsystems mark all buses. To accelerate the speed of convergence and prevent the label oscillation, this paper proposes a game strategy of cooperation between buses and subsystems by evaluating the Shapley value of buses. The partitioning constraints are integrated into each label propagation process to assess the feasibility of the partitioning strategy. Finally, case studies on the IEEE 39- and 118-bus test systems and an actual urban power system in South China verify the effectiveness of the proposed method.

Evaluation of machine learning models for cytochrome P450 3A4, 2D6, and 2C9 inhibition.

Cytochrome P450 (CYP) enzymes are involved in the metabolism of approximately 75% of marketed drugs. Inhibition of the major drug-metabolizing P450s could alter drug metabolism and lead to undesirable drug-drug interactions. Therefore, it is of great significance to explore the inhibition of P450s in drug discovery. Currently, machine learning including deep learning algorithms has been widely used for constructing in silico models for the prediction of P450 inhibition. These models exhibited varying predictive performance depending on the use of machine learning algorithms and molecular representations. This leads to the difficulty in the selection of appropriate models for practical use. In this study, we systematically evaluated the conventional machine learning and deep learning models for three major P450 enzymes, CYP3A4, CYP2D6, and CYP2C9 from several perspectives, such as algorithms, molecular representation, and data partitioning strategies. Our results showed that the XGBoost and CatBoost algorithms coupled with the combined fingerprint/physicochemical descriptor features exhibited the best performance with Area Under Curve (AUC) of 0.92, while the deep learning models were generally inferior to the conventional machine learning models (average AUC reached 0.89) on the same test sets. We also found that data volume and sampling strategy had a minor effect on model performance. We anticipate that these results are helpful for the selection of molecular representations and machine learning/deep learning algorithms in the P450 model construction and the future model development of P450 inhibition.

Exploiting Domain Partition in Response Function-Based Dynamic Surrogate Modeling: A Continuous Crystallizer Study

Given the exponential rise in the amount of data requiring processing in all engineering fields, phenomenological models have become computationally cumbersome. For this reason, more efficient data-driven models have been recently used with the purpose of substantially reducing simulation computational times. However, especially in process engineering, the majority of the proposed surrogate models address steady-state problems, while poor studies refer to dynamic simulation modeling. For this reason, using a response function-based approach, a crystallization unit case study was set up in order to derive a dynamic data-driven model for crystal growth whose characteristic differential parameters are derived via Response Surface Methodology. In particular, multiple independent variables were considered, and a well-established sampling technique was exploited for sample generation. Then, different sample sizes were tested and compared in terms of accuracy indicators. Finally, the domain partition strategy was exploited in order to show its relevant impact on the final model accuracy. In conclusion, the outcome of this study proved that the proposed procedure is a suitable methodology for dynamic system metamodeling, as it shows good compliance and relevant improvement in terms of computational time. In terms of future research perspectives, testing the proposed procedure on different systems and in other research fields would allow for greater improvement and would, eventually, extend its validity.

Performance modeling and running strategy of parallel cdugksFOAM program

The cdugksFOAM program realizes the physical space grid and the velocity space grid in parallel at the same time. Its distinguishing feature lies in its potential for large-scale parallelism. However, the running time of the cdugksFOAM program is significantly dependent on the number of physical and velocity space partitions. In order to find the optimal partitioning strategies for a specific CFD problem running on a parallel computer, we performed performance modeling of the cdugksFOAM program. Firstly, we proposed a floating-point operations model, a MPI communication volume model, and a memory consumption model. Based on these models, we established a Roofline model to predict the computational time, and a model to predict communication time. According to the computational time model and the communication time model, the execution time model was proposed and its effectiveness was verified with two cases. Finally, the optimal running strategy that minimizes the product of the number of computing nodes and execution time was identified, providing meaningful guidance for the economic execution of the program.

PreRoutGNN for Timing Prediction with Order Preserving Partition: Global Circuit Pre-training, Local Delay Learning and Attentional Cell Modeling

Pre-routing timing prediction has been recently studied for evaluating the quality of a candidate cell placement in chip design. It involves directly estimating the timing metrics for both pin-level (slack, slew) and edge-level (net delay, cell delay), without time-consuming routing. However, it often suffers from signal decay and error accumulation due to the long timing paths in large-scale industrial circuits. To address these challenges, we propose a two-stage approach. First, we propose global circuit training to pre-train a graph auto-encoder that learns the global graph embedding from circuit netlist. Second, we use a novel node updating scheme for message passing on GCN, following the topological sorting sequence of the learned graph embedding and circuit graph. This scheme residually models the local time delay between two adjacent pins in the updating sequence, and extracts the lookup table information inside each cell via a new attention mechanism. To handle large-scale circuits efficiently, we introduce an order preserving partition scheme that reduces memory consumption while maintaining the topological dependencies. Experiments on 21 real world circuits achieve a new SOTA R2 of 0.93 for slack prediction, which is significantly surpasses 0.59 by previous SOTA method. Code will be available at: https://github.com/Thinklab-SJTU/EDA-AI.

Framework for Agent-Based Multistage Application Partitioning Algorithm in Mobile Cloud Computing

The idea of computational offloading is quickly catching on in the world of mobile cloud computing (MCC). Today’s applications have heavy demands on power and computing resources, creating issues with energy consumption, storage capacity, and mobile device performance. Mobile devices may efficiently offload their calculations to cloud servers using the offloading paradigm and then get the processed results back onto the device. The investigation relates to identifying the specific application components that should be offloaded and run remotely and which parts are supposed to be treated locally. The applications need to be partitioned to differentiate between remote and local codes. In this paper, an agent-based multistage graph partitioning (ABMP) scheme is proposed. The framework of the scheme is based on three-tier architecture that includes mobile, cloudlet, and cloud for the execution of application tasks. The main goal is to provide an efficient partitioning and offloading scheme in the mobile cloud computing area. The results show that incorporating both agent-based multistage graph partitioning and offloading algorithms yields superior performance as compared to previous methods in terms of reducing execution costs and conserving battery life for mobile devices.

Data Communication Algorithm of HPDB Parallel Database System Based on Computer Network

High-Performance Database (HPDB) is designed to meet the demands of today’s modern data-driven world, as massive volumes of information need to be accessed and analyzed with minimal latency. A pivotal aspect of their operation lies in efficient data communication among nodes in a computer network, which is essential for parallel database systems. HPDBs may involve distributed architectures and parallel database systems to store and process data across multiple nodes or servers in a network. Hence, an algorithm called Hybridized Partitioning Strategy (HPS)-based Communication (C) for achieving HPDB has been proposed to facilitate data transmission and coordination across a computer network using the Message Passing Interface (MPI) protocol. The proposed HPS-C-HPDB technique includes partitioning and distributing data, query routing, and load balancing strategies to achieve high-performance levels. The (HPS) combines hash and range partitioning methods for effective processing and retrieval to balance data distribution, reduce communication overhead in parallel databases, and improve the system’s performance. In parallel database systems, query routing effectively routes requests into the optimal nodes or partitions based on the query’s conditions and the data’s placement and guarantees efficient data processing and retrieval. The proposed scheme is evaluated using various performance metrics like throughput, response time, speedup, and communication overhead analysis.

Application of Distributed Graphs for Facilitation of Scalable Botnet Detection and Response

Botnets pose a significant threat to modern network environments, exploiting compromised devices to carry out malicious activities such as distributed denial-of-service attacks, spam campaigns, and data theft. Traditional centralized detection systems often struggle to handle the scale and complexity of botnet attacks, leading to delays in detection and response. In response to these challenges, this paper explores the application of distributed graphs for facilitating scalable botnet detection and response. Distributed graphs offer a promising approach for modelling and analyzing complex network structures, enabling efficient detection of botnet propagation patterns and anomalous behaviour across distributed computing environments. The paper presents an overview of distributed graph-based botnet detection systems, discussing their architecture, design considerations, and key concepts such as graph partitioning, vertex-centric computation, and message passing in distributed graph algorithms. Case studies illustrate the practical application of distributed graph-based botnet detection in diverse network environments, highlighting success stories, challenges encountered, and lessons learned from deploying distributed graph systems in production cybersecurity operations. Finally, the paper discusses challenges and open research questions in the field of distributed graph-based botnet detection, addressing issues such as graph partitioning strategies, fault tolerance, privacy-preserving techniques, and integration with other security tools. It proposes potential avenues for future research and development in scalable botnet detection using distributed graphs, emphasizing the importance of adaptive threat response, collaboration with industry partners, and continuous improvement in detection algorithms for enhancing cybersecurity resilience against botnet attacks.

GraphScale: Scalable Processing on FPGAs for HBM and Large Graphs

Recent advances in graph processing on FPGAs promise to alleviate performance bottlenecks with irregular memory access patterns. Such bottlenecks challenge performance for a growing number of important application areas like machine learning and data analytics. While FPGAs denote a promising solution through flexible memory hierarchies and massive parallelism, we argue that current graph processing accelerators either use the off-chip memory bandwidth inefficiently or do not scale well across memory channels. In this work, we propose GraphScale, a scalable graph processing framework for FPGAs. GraphScale combines multi-channel memory with asynchronous graph processing (i.e., for fast convergence on results) and a compressed graph representation (i.e., for efficient usage of memory bandwidth and reduced memory footprint). GraphScale solves common graph problems like breadth-first search, PageRank, and weakly connected components through modular user-defined functions, a novel two-dimensional partitioning scheme, and a high-performance two-level crossbar design. Additionally, we extend GraphScale to scale to modern high-bandwidth memory (HBM) and reduce partitioning overhead of large graphs with binary packing.

SWattention: designing fast and memory-efficient attention for a new Sunway Supercomputer

In the past few years, Transformer-based large language models (LLM) have become the dominant technology in a series of applications. To scale up the sequence length of the Transformer, FlashAttention is proposed to compute exact attention with reduced memory requirements and faster execution. However, implementing the FlashAttention algorithm on the new generation Sunway Supercomputer faces many constraints such as the unique heterogeneous architecture and the limited memory bandwidth. This work proposes SWattention, a highly efficient method for computing the exact attention on the SW26010pro processor. To fully utilize the 6 core groups (CG) and 64 cores per CG on the processor, we design a two-level parallel task partition strategy. Asynchronous memory access is employed to ensure that memory access overlaps with computation. Additionally, a tiling strategy is introduced to determine optimal SRAM block sizes. Compared with the standard attention, SWattention achieves around 2.0x speedup for FP32 training and 2.5x speedup for mixed-precision training. The sequence lengths range from 1k to 8k and scale up to 16k without being out of memory. As for the end-to-end performance, SWattention achieves up to 1.26x speedup for training GPT-style models, which demonstrates that SWattention enables longer sequence length for LLM training.

Optimizing austenite and carbide content in medium carbon silicon-rich steel: A stepped partitioning strategy and analysis of strengthening and ductility enhancement mechanisms

To achieve the objective of enhancing the yield strength (YS) of quenching and partitioning (Q&P) steel without compromising tensile strength and ductility, a stepped partitioning (SP) strategy was proposed for Q&P treatment on medium carbon silicon-rich steel. The optimized partitioning strategy involves a two-stage partitioning, the first stage is conducted at temperatures above the martensite start (Ms) temperature, followed by the second stage carried out at a lower temperature for an extended period, involving isothermal partitioning. Experimental results demonstrate that the SP strategy yields a significant quantity of small carbides, with an average size of 59.02 nm, along with 15.71 vol% of retained austenite (RA) and a carbon concentration of 1.01 wt% in RA. Furthermore, the SP-treated specimen exhibits a remarkable balance of strength and ductility, showcasing YS, ultimate tensile strength (UTS), and total elongation (TE) values reaching 1732 MPa, 2263 MPa, and 14.9%, respectively. The microstructure evolution during tensile deformation demonstrates that RA in short-duration conventional partitioning specimens underwent premature martensitic transformation during tensile deformation due to early deformation, whereas RA in long time conventional partitioning specimens was overstabilized due to highest C concentration, which resulted in poor enhancement of strengthening and ductility by TRIP effect. In contrast, the SP strategy achieved a reasonable stability of RA, resulting in a simultaneous increase of tensile strength and ductility. Simultaneously, an analysis of precipitation strengthening indicates that the abundance of fine carbides in martensite contributes to an increase in YS due to precipitation strengthening.