Data Partitioning Method Research Articles

A Lightweight Deep Learning Model for Forecasting the Fishing Ground of Purpleback Flying Squid (Sthenoteuthis oualaniensis) in the Northwest Indian Ocean

The purpleback flying squid (Sthenoteuthis oualaniensis) is an economically significant cephalopod species in the Northwest Indian Ocean. Predicting its fishing grounds can provide a crucial foundation for fishery management and production. In this research, we collected data from China’s light-purse seine fishery in the Northwest Indian Ocean from 2016 to 2020 to train and validate the AlexNet and VGG11 models. We designed a data partitioning method (DPM) to divide the training set into three scenarios, namely DPM-S1, DPM-S2, and DPM-S3. Firstly, DPM-S1 was employed to select the base model (BM). Subsequently, the optimal BM was lightweighted to obtain the optimal model (OM). The OM, known as the AlexNetMini model, has a model size that is one-third of that of the BM-AlexNet model. Our results also showed the following: (1) the F1-scores for AlexNet and AlexNetMini across the datasets DPM-S1, -S2, and -S3 were 0.6957, 0.7505, and 0.7430 for AlexNet and 0.6992, 0.7495, and 0.7486 for AlexNetMini, suggesting that both models exhibited comparable predictive performance; (2) the optimal dropout values for the AlexNetMini model were 0 and 0.2, and the optimal training set proportion was 0.8; (3) AlexNetMini utilized both DPM-S2 and DPM-S3, yielding comparable outcomes. However, given that the training duration for DPM-S3 was relatively shorter, DPM-S3 was selected as the preferred method for data partitioning. The findings of our study indicated that the lightweight model for the purpleback flying squid fishing ground prediction, specifically AlexNetMini, demonstrated superior performance compared to the original AlexNet model, particularly in terms of efficiency. Our study on the lightweight method for deep learning models provided a reference for enhancing the usability of deep learning in fisheries.

Mitigating Data Leakage in a WiFi CSI Benchmark for Human Action Recognition.

Human action recognition using WiFi channel state information (CSI) has gained attention due to its non-intrusive nature and potential applications in healthcare, smart environments, and security. However, the reliability of methods developed for CSI-based action recognition is often contingent on the quality of the datasets and evaluation protocols used. In this paper, we uncovered a critical data leakage issue, which arises from improper data partitioning, in a widely used WiFi CSI benchmark dataset. Specifically, the benchmark fails to separate individuals between the training and test sets, leading to inflated performance metrics as models inadvertently learn individual-specific features rather than generalizable action patterns. We analyzed this issue in depth, retrained several benchmarked models using corrected data partitioning methods, and demonstrated a significant drop in accuracy when individuals were properly separated across training and testing. Our findings highlight the importance of rigorous data partitioning in CSI-based action recognition and provide recommendations for mitigating data leakage in future research. This work contributes to the development of more robust and reliable human action recognition systems using WiFi CSI.

A Recommendation System for Trigger-Action Programming Rules via Graph Contrastive Learning.

Trigger-action programming (TAP) enables users to automate Internet of Things (IoT) devices by creating rules such as "IF Device1.TriggerState is triggered, THEN Device2.ActionState is executed". As the number of IoT devices grows, the combination space between the functions provided by devices expands, making manual rule creation time-consuming for end-users. Existing TAP recommendation systems enhance the efficiency of rule discovery but face two primary issues: they ignore the association of rules between users and fail to model collaborative information among users. To address these issues, this article proposes a graph contrastive learning-based recommendation system for TAP rules, named GCL4TAP. In GCL4TAP, we first devise a data partitioning method called DATA2DIV, which establishes cross-user rule relationships and is represented by a user-rule bipartite graph. Then, we design a user-user graph to model the similarities among users based on the categories and quantities of devices that they own. Finally, these graphs are converted into low-dimensional vector representations of users and rules using graph contrastive learning techniques. Extensive experiments conducted on a real-world smart home dataset demonstrate the superior performance of GCL4TAP compared to other state-of-the-art methods.

L-FNNG: Accelerating Large-Scale KNN Graph Construction on CPU-FPGA Heterogeneous Platform

Due to the high complexity of constructing exact k -nearest neighbor graphs, approximate construction has become a popular research topic. The NN-Descent algorithm is one of the representative in-memory algorithms. To effectively handle large datasets, existing state-of-the-art solutions combine the divide-and-conquer approach and the NN-Descent algorithm, where large datasets are divided into multiple partitions, and a subgraph is constructed for each partition before all the subgraphs are merged, reducing the memory pressure significantly. However, such solutions fail to address inefficiencies in large-scale k -nearest neighbor graph construction. In this paper, we propose L-FNNG, a novel solution for accelerating large-scale k -nearest neighbor graph construction on CPU-FPGA heterogeneous platform. The CPU is responsible for dividing data and determining the order of partition processing, while the FPGA executes all construction tasks to utilize the acceleration capability fully. To accelerate the execution of construction tasks, we design an efficient FPGA accelerator, which includes the Block-based Scheduling (BS) and Useless Computation Aborting (UCA) techniques to address the problems of memory access and computation in the NN-Descent algorithm. We also propose an efficient scheduling strategy that includes a KD-tree-based data partitioning method and a hierarchical processing method to address scheduling inefficiency. We evaluate L-FNNG on a Xilinx Alveo U280 board hosted by a 64-core Xeon server. On multiple large-scale datasets, L-FNNG achieves, on average, 2.3× construction speedup over the state-of-the-art GPU-based solution.

GCTNet: a graph convolutional transformer network for major depressive disorder detection based on EEG signals

Objective. Identifying major depressive disorder (MDD) using objective physiological signals has become a pressing challenge. Approach. Hence, this paper proposes a graph convolutional transformer network (GCTNet) for accurate and reliable MDD detection using electroencephalogram (EEG) signals. The developed framework integrates a residual graph convolutional network block to capture spatial information and a Transformer block to extract global temporal dynamics. Additionally, we introduce the contrastive cross-entropy (CCE) loss that combines contrastive learning to enhance the stability and discriminability of the extracted features, thereby improving classification performance. Main results. The effectiveness of the GCTNet model and CCE loss was assessed using EEG data from 41 MDD patients and 44 normal controls, in addition to a publicly available dataset. Utilizing a subject-independent data partitioning method and 10-fold cross-validation, the proposed method demonstrated significant performance, achieving an average Area Under the Curve of 0.7693 and 0.9755 across both datasets, respectively. Comparative analyses demonstrated the superiority of the GCTNet framework with CCE loss over state-of-the-art algorithms in MDD detection tasks. Significance. The proposed method offers an objective and effective approach to MDD detection, providing valuable support for clinical-assisted diagnosis.

In situ monitoring with melt pool data based on multi-signal fusion method in laser powder bed fusion

In situ monitoring technologies for additive manufacturing are severely affected by signal-to-defect spatial registration, and a single monitoring solution that captures a specific process signature cannot identify defects with sufficient accuracy in industrial applications. In this paper, a data partitioning method was proposed to reduce registration errors and facilitate the extraction of defect-related representative features from data segments. Then, three multi-signal fusion methods based on convolutional neural networks were established from feature-level fusion, decision-level fusion, and data-level fusion, which enhanced the signal-to-defect correlation by means of complementary sensor information. The results showed that the feature-level fusion obtained the optimal classification performance compared with the other two fusion strategies. Among them, the classification accuracies of 100-, 150-, 200-, 300-, 400-, and 500-μm defects reached 74.41%, 89.84%, 90.82%, 93.95%, 97.85%, and 98.83%, respectively. The proposed data fusion approach outperformed the individual signal-based models for quality classification.

EFNN-Nul0- a trustworthy knowledge extraction about stress identification through evolving fuzzy neural networks

This paper presents a novel hybrid architecture, denoted as EFNN-Nul0 (evolving neuro-fuzzy system based on null-unineurons), meticulously crafted for stress identification within the realm of pattern classification. The model seamlessly integrates neural networks, fuzzy systems, and n-uninorms to cultivate knowledge within an adaptive, real-time learning context. Neuro-fuzzy rules, encapsulating interdependencies among input features through IF-THEN type relationships, are formulated utilizing null-unineurons derived from n-uninorms. This construction enables the articulation of both AND- and OR-connections, thereby augmenting the interpretability of the generated rules. The evolution of neurons is facilitated by an extended adaptation of the autonomous data partition method (ADPA). To dynamically interpret the evolution of rules, the paper introduces (i) a mechanism that tracks changes in rules over data stream samples, providing insights into the process dynamics as a structural active learning component, and (ii) a concept for incrementally updating feature weights. These weights encapsulate the varying impact levels of features on stress identification, enabling the reduction of rule length by effectively masking out less significant features. The outcomes of the rules are represented by certainty vectors, recursively updated through an indicator-based recursive weighted least squares (I-RWLS) approach. Neuron activation levels play a pivotal role in determining the weights, ensuring stable local learning. The effectiveness of the proposed model is substantiated through a comprehensive comparison with existing hybrid and evolving approaches in the literature, focusing on binary and multi-class pattern classification. Aspects that identify special characteristics of neuro-fuzzy models related to interpretability will also be demonstrated in this work. The results demonstrate the model's superior performance, characterized by consistently higher accuracy trend lines over time. Furthermore, the model's interpretability is underscored by the coherent neuro-fuzzy rules, contributing to its remarkable accuracy (the EFNN-Nul0 achieved approximately 99% accuracy in stress identification), establishing its efficacy in addressing stress identification within classification problems.

Farm electricity system simulator (FESS): A platform for simulating electricity utilisation on dairy farms

The objective of this paper was to define, validate and demonstrate a model capable of accurately simulating dairy farm electricity consumption across varying herd and parlour sizes, to facilitate research investigating renewable energy systems (RES) and demand side management (DSM). The Farm Electricity System Simulator (FESS) was developed using grey-box modelling techniques utilizing empirical data for parameter tuning. Empirical data were gathered from nine spring calving, pasture based dairy farms located in the Republic of Ireland. A k-means clustering analysis was conducted, separating the farms into three, near homogenous groups, from which representative farms were selected. FESS was trained using 12 months of data from three representative farms using the repeat hold out method for data partitioning with 75 % of data used for training and 25 % used for validation. An optimisation algorithm was used to minimize the error during model training. Through cross-validation, FESS achieved a root mean squared error (RMSE) of 7.65 kWh, mean absolute percentage error (MAPE) of 7.10 %, mean percentage error (MPE) of −0.86 % and a relative prediction error (RPE) of 7.56 % for total daily electricity consumption. Across the three farms, the simulated outputs of FESS achieved an average R2 value of 0.72, demonstrating good agreement with observed data. FESS’s utility was demonstrated by analysing the effects of different electricity pricing structures and on-site solar photovoltaic electricity generation on total farm energy costs. We concluded that FESS simulated on-farm electricity consumption with sufficient accuracy for the intended application. FESS accurately simulated dairy farm electricity consumption across three dairy farms of different herd and parlour sizes while evaluating the effects of demand side management and renewable generation on farm electricity consumption and costs.

Accelerating multivariate functional approximation computation with domain decomposition techniques

Modeling large datasets through Multivariate Functional Approximations (MFA) provide an elegant way to handle many visualization and scientific analysis workflows. The process necessitates scalable data partitioning methods to compute MFA representations efficiently without compromising the accuracy or continuity of the reconstructed solution. We propose a domain-decomposed method for computing the MFA with B-spline bases, which reduces the total work per task and uses a restricted Additive Schwarz (RAS) method to converge the control point data degrees-of-freedom along subdomain boundaries. We provide an in-depth analysis of the parallel approach with domain decomposition solvers, aiming to minimize local subdomain error residuals and recover high-order continuity at subdomain interfaces with appropriate choices of knot overlaps. The communication cost, determined by the overlap regions in the RAS implementation, is optimized to recover the numerical error profile of the single subdomain case. Our proposed method stands in contrast to previous methods, which typically only recover either C0 or at best C1 continuity for arbitrary B-spline degree expansions, or those that require post-processing to blend discontinuities in the reconstructed data. We demonstrate the effectiveness of our approach using analytical and real-world datasets in 1D, 2D, and 3D through both strong and weak scaling studies. The performance results indicate that the overall cost of computing the approximation is directly proportional to the underlying nearest-neighbor communication implementation, and is only weakly dependent on the overlap region size that determines the size of the messages. This finding underscores the efficiency and scalability of our proposed method, making it a promising solution for handling large datasets in scientific workflows.

Multicore Parallelized Spatial Overlay Analysis Algorithm Using Vector Polygon Shape Complexity Index Optimization

As core algorithms of geographic computing, overlay analysis algorithms typically have computation-intensive and data-intensive characteristics. It is highly important to optimize overlay analysis algorithms by parallelizing the vector polygons after reasonable data division. To address the problem of unbalanced data partitioning in the task decomposition process for parallel polygon overlay analysis and calculation, this paper presents a data partitioning method based on shape complexity index optimization, which achieves data equalization among multicore parallel computing tasks. Taking the intersection operator and difference operator of the Vatti algorithm as examples, six polygon shape indexes are selected to construct the shape complexity model, and the vector data are divided in accordance with the calculated shape complexity results. Finally, multicore parallelism is achieved based on OpenMP. The experimental results show that when a data set with a large amount of data is used, the effect of the multicore parallel execution of the Vatti algorithm’s intersection operator and difference operator based on shape complexity division is clearly improved. With 16 threads, compared with the serial algorithm, speedups of 29 times and 32 times can be obtained. Compared with the traditional multicore parallel algorithm based on polygon number division, the speed can be improved by 33% and 29%, and the load balancing index is reduced. For a data set with a small amount of data, the acceleration effect of this method is similar to that of traditional methods involving multicore parallelism.

MA-CapsNet-DA: Speech emotion recognition based on MA-CapsNet using data augmentation

Speech emotion recognition (SER) plays a crucial role in Human–computer interaction (HCI) applications. However, it has two challenges: the lack of effectiveness of deep learning models and data scarcity issues. As a result, the deep learning models used in SER would suffer from overfitting seriously. In this study, a novel SER model called Max-avg-pooling capsule network (MA-CapsNet) is proposed and it is an improved capsule network customized for SER. We also adopt Data augmentation (DA) techniques to tackle the data scarcity issue. The proposed MA-CapsNet model consists of five sequential modules: conv-max-pooling, conv-avg-pooling, convolution, primary capsule, and digital capsule module. Furthermore, a new evaluation metric called the Expected accuracy index (EAI) is presented to evaluate the model performance effectively. The proposed approach demonstrates strong advantages over its peer models under different data partition methods, especially for augmented datasets. Experimental results also show that the proposed model has good interpretability in comparison to the peer methods for its less complicated learning topology, relatively smaller parameter sets, and fewer input features.

A parallel strategy to accelerate neighborhood operation for raster data coordinating CPU and GPU

ABSTRACT This study presents an asynchronous parallel strategy coordinating central processing unit (CPU) and graphic processing unit (GPU) to accelerate neighborhood operation (NO). Specifically, we propose a data partitioning method called multi-anchor task queuing and a task scheduling method called bi-direction task scheduling, which can support CPU and GPU to find the responsible data blocks rapidly and concurrently handle their tasks via a bi-direction merge. Moreover, we optimize the organization of threads distributed among the CPU and GPU. Experimental results show that when a 1.7 GB raster dataset is processed, the speedup ratio achieved by the proposed parallel algorithm reaches 29.63, which is 19% and 18% higher than those of the GPU and standard asynchronous parallel algorithm, respectively. Additionally, the load balance index is below 0.085, which is significantly better than the value achieved by a conventional algorithm. Thus, the strategy achieves a higher speedup ratio and more adaptable load balance, thereby accelerating the NO more efficiently. Further, the impacts of the data volume, computational intensity, organization mode of the GPU threads, and granularity of the GPU stream on the parallel efficiency are evaluated and discussed. We also test the efficiency of four other common NOs with our strategy.

Estimation of Parameters With Partitioned Data From Large Amplitude Maneuvers of an Aircraft

In this paper data partitioning method is applied to concatenated Large amplitude manoeuvres (LAM) of a fighter aircraft to estimate its stabiLity and control derivatives. Stepwise multiple linear regression (SMLR) is utilized to determine the model structure within each partitioned data set. It is shown that the procedure with data generated from a research simulator, yields a useful method for estimating the pitch up tendency exhibited b), the aircraft during certain angle of attack (AOA) range. The SMLR is also applied to estimate non-linear aerodynamic derivatives of the aircraft from LAM data. The method is applied to reaL LAM data of the aircraft and gives reasonably consistent estimates of the aerodynamic derivatives. Results using model error method for the longitudinal derivatives from LAMs are also presented.

Physics-informed deep learning to forecast {widehat{{varvec{M}}}}_{{varvec{m}}{varvec{a}}{varvec{x}}} during hydraulic fracturing

Short-term forecasting of estimated maximum magnitude ({widehat{M}}_{max}) is crucial to mitigate risks of induced seismicity during fluid stimulation. Most previous methods require real-time injection data, which are not always available. This study proposes two deep learning (DL) approaches, along with two data-partitioning methods, that rely solely on preceding patterns of seismicity. The first approach forecasts {widehat{M}}_{max} directly using DL; the second incorporates physical constraints by using DL to forecast seismicity rate, which is then used to estimate {widehat{M}}_{max}. These approaches are tested using a hydraulic-fracture monitoring dataset from western Canada. We find that direct DL learns from previous seismicity patterns to provide an accurate forecast, albeit with a time lag that limits its practical utility. The physics-informed approach accurately forecasts changes in seismicity rate, but sometimes under- (or over-) estimates {widehat{M}}_{max}. We propose that significant exceedance of {widehat{M}}_{max} may herald the onset of runaway fault rupture.

Improving Structured Grid-Based Sparse Matrix-Vector Multiplication and Gauss–Seidel Iteration on GPDSP

Structured grid-based sparse matrix-vector multiplication and Gauss–Seidel iterations are very important kernel functions in scientific and engineering computations, both of which are memory intensive and bandwidth-limited. GPDSP is a general purpose digital signal processor, which is a very significant embedded processor that has been introduced into high-performance computing. In this paper, we designed various optimization methods, which included a blocking method to improve data locality and increase memory access efficiency, a multicolor reordering method to develop Gauss–Seidel fine-grained parallelism, a data partitioning method designed for GPDSP memory structures, and a double buffering method to overlap computation and access memory on structured grid-based SpMV and Gauss–Seidel iterations for GPDSP. At last, we combined the above optimization methods to design a multicore vectorization algorithm. We tested the matrices generated with structured grids of different sizes on the GPDSP platform and obtained speedups of up to 41× and 47× compared to the unoptimized SpMV and Gauss–Seidel iterations, with maximum bandwidth efficiencies of 72% and 81%, respectively. The experiment results show that our algorithms could fully utilize the external memory bandwidth. We also implemented the commonly used mixed precision algorithm on the GPDSP and obtained speedups of 1.60× and 1.45× for the SpMV and Gauss–Seidel iterations, respectively.

SLR: A million-scale comprehensive crossword dataset for simultaneous learning and reasoning

The progress of the natural language understanding (NLU) community has put forward higher demands for the knowledge reserve and reasoning ability of the model. However, existing schemes for improving model capabilities often split them into two separate tasks. The task of enabling models to learn and reason about knowledge simultaneously has not received sufficient attention. In this paper, we propose a novel crossword-based NLU task that imparts knowledge information to a model by solving crossword clues and simultaneously trains the model to infer new knowledge from existing knowledge. To this end, we construct a comprehensive crossword dataset SLR containing more than 4 million unique clue-answer pairs. Compared to existing crossword datasets, SLR is more comprehensive and contains linguistic knowledge, expertise in various fields, and commonsense knowledge. Meanwhile, to evaluate the reasoning ability of the model, we design clever details in the reasoning of the answers, most clues require the solver to reason through two or more pieces of knowledge to arrive at an answer. We analyze the composition of the dataset and the similarities and differences of the various types of clues via sampling and consider various data partitioning methods to enhance the generalization ability of the training set. Furthermore, we test the performance of several different advanced models and methods on this dataset and analyze the strengths and weaknesses of each. An interesting conclusion is that even powerful language models perform poorly on tasks that require reasoning.

A Clustering Algorithm for Tunnel Boring Machine Data Based on Ridge Regression and Fuzzy C-Means

A fuzzy clustering data partitioning method based on ridge regression is proposed to address the high correlation between the data attributes of the tunnel boring machine. The method utilizes the fuzzy partition matrix as the weight for ridge regression analysis and then employs the resulting regression equation as the clustering model to iteratively approach the target minimum value. This process enables data clustering based on the relevant characteristics of the data attributes. Experimental results using functional data sets demonstrate that this method achieves higher accuracy in clustering functional correlation data compared to traditional methods. Additionally, the proposed algorithm is evaluated using measured data from a bid section of a city subway. Results indicate that the algorithm effectively addresses the problem of data classification and key performance index prediction, with a misclassification rate and prediction error of 23.8% and 2.11%, respectively. These findings meet the engineering requirements and provide support for the subsequent data analysis of the tunnel boring machine.

Analysis of the of training and test data distribution for audio series classification

The effectiveness of machine learning algorithms for any given task largely depends on the training and test datasets. This manifests itself not only in the amount of data, but also in its content (that is, its relevance for the task at hand), as well as in its organization. Generally, the common approach is to split the dataset into training and testing sets to avoid model overfitting. In addition, to achieve better metrics for the selected criteria (accuracy, learning rate, etc.) of model performance, different ratios of training and test sets are used in the partitioning. The goal of this paper is to analyze methods of data set partitioning for use in training neural networks and statistical models. One of the reviewed methods, specifically the cross-validation method, was applied to a dataset developed from the LibriSpeach corpus, an open English speech corpus based on the LirbiVox project of voluntarily contributed audio books. The result of applying the selected data partitioning method on the selected data set is demonstrated.

A Novel Data Partitioning Method for Active Privacy Protection Applied to Medical Records

In recent years, cloud computing has attracted extensive attention from industry and academia due to its convenience and ubiquity. As a new Internet-based IT service model, cloud computing has brought revolutionary changes to traditional computing and storage services. More and more individual users and enterprises are willing to deploy their own data and applications on the cloud platform, but the accompanying security issues have also become an obstacle to the development of cloud computing. Multi-tenancy and virtualization technologies are the main reasons why cloud computing faces many security problems. Through the virtualization of storage resources, multi-tenant data are generally stored as shared physical storage resources. To distinguish the data of different tenants, labels are generally used to distinguish them. However, this simple label cannot resist the attack of a potential malicious tenant, and data still has the risk of leakage. Based on this, this paper proposed a data partitioning method in a multi-tenant scenario to prevent privacy leakage of user data. We demonstrate the use of the proposed approach in protecting patient data in medical records in health informatics. Experiments show that the proposed algorithm can partition the attributes more fine-grained and effectively protect the sensitive information in the data.

Clustering-based method for big spatial data partitioning

“Internet of Things” (IoT) is considered one of the main focus areas of research in computer systems and networks. Since IoT devices are installed in static geographic places or on board a moving trackable object, the data generated by the device is mainly characterized as spatial data. The spatial data generated by IoT devices scale up in volume, velocity, and veracity so they tend to be considered “Big Data”. “Big Spatial Data” requires the development of special frameworks that use state-of-the-art technologies in data storage, query, and analysis. These frameworks are mainly characterized by the use of a parallel programming model that partitions the spatial data into smaller chunks that can be handled in parallel. The development of an optimal method for spatial data partitioning is essential in implementing such systems.In this paper, we propose, design, and implement a new method for spatial data partitioning based on K-Means clustering, an unsupervised machine learning algorithm. The design is based on a well-defined conceptual, mathematical, and programming model for a general spatial data partitioning method. The main component of the suggested model is an implementation of K-Means clustering suited for spatial data.The new method is designed, implemented, and tested to prove its ability to achieve partitioning objectives and the efficiency of its performance. The results of the tests are benchmarked against one of the most widely adopted approaches in partitioning spatial data and prove the ability of the novel method to surpass it in some of the evaluation criteria.