Data Partitioning Process Research Articles

Hyperparameter Tuning of Machine Learning Algorithms Using Response Surface Methodology: A Case Study of ANN, SVM, and DBN

This study applies response surface methodology (RSM) to the hyperparameter fine-tuning of three machine learning (ML) algorithms: artificial neural network (ANN), support vector machine (SVM), and deep belief network (DBN). The purpose is to demonstrate RSM effectiveness in maintaining ML algorithm performance while reducing the number of runs required to reach effective hyperparameter settings in comparison with the commonly used grid search (GS). The ML algorithms are applied to a case study dataset from a food producer in Thailand. The objective is to predict a raw material quality measured on a numerical scale. K-fold cross-validation is performed to ensure that the ML algorithm performance is robust to the data partitioning process in the training, validation, and testing sets. The mean absolute error (MAE) of the validation set is used as the prediction accuracy measurement. The reliability of the hyperparameter values from GS and RSM is evaluated using confirmation runs. Statistical analysis shows that (1) the prediction accuracy of the three ML algorithms tuned by GS and RSM is similar, (2) hyperparameter settings from GS are 80% reliable for ANN and DBN, and settings from RSM are 90% and 100% reliable for ANN and DBN, respectively, and (3) savings in the number of runs required by RSM over GS are 97.79%, 97.81%, and 80.69% for ANN, SVM, and DBN, respectively.

IPC: Resource and network cost-aware distributed stream scheduling on skewed streams

The performance of distributed stream processing engines is significantly compromised when processing stream data with skewed distribution. Current stream partitioning schemes are not able to meet the rigorous requirements of distributed stream processing. We show that network cost is an essential factor for partitioning data, and this factor should be considered when designing a stream partitioning scheme. Additionally, we should efficiently utilize resources in the data partitioning process. Current stream partitioning schemes either use a shuffle grouping approach that efficiently manages workload but faces scalability issues in terms of memory or uses hash-based key grouping schemes that suffer from load balancing issues.We argue that network cost and resource utilization are two crucial factors for stream partitioning schemes. We propose and implement a distributed stream partitioning scheme call IPC that minimizes the network cost and efficiently utilizes resources by leveraging two techniques: process near source and process at local. It also utilizes key splitting and local load estimation techniques to achieve load balancing. We implement the IPC on top of Apache Storm. Experiment results using large scale real-time datasets show that IPC achieves an up to 4.2x improvement in throughput and reduces processing latency by 97% compared to state-of-the-art designs.

BEST: a decision tree algorithm that handles missing values

The main contribution of this paper is the development of a new decision tree algorithm. The proposed approach allows users to guide the algorithm through the data partitioning process. We believe this feature has many applications but in this paper we demonstrate how to utilize this algorithm to analyse data sets containing missing values. We tested our algorithm against simulated data sets with various missing data structures and a real data set. The results demonstrate that this new classification procedure efficiently handles missing values and produces results that are slightly more accurate and more interpretable than most common procedures without any imputations or pre-processing.

Variance analysis of software ageing problems

Software ageing problems are mainly caused by resource consumption exhaustion, so many researchers focused on predicting software resource consumption. However, the loss analysis using variance has not been done. In this study, the authors propose a framework to analyse variance change in the resource consumption prediction problems. This framework is made up of three steps. First, an original variance decomposition is proposed in view of data sampling and partitioning process. Second, in order to study the influence of data sampling and partitioning process to the variance, the enhanced Friedman test plus Nemenyi post-hoc test is introduced. Lastly, they propose a corrected t-test to analyse the performances of two regression algorithms: auto-regressive integrated moving average and artificial neuron network. In the experiments, they analyse the variance in two levels: operating system level and application level. They find the result that k is equal to ten for k-fold cross-validation is proper for resource consumption prediction, although the contribution to variance is almost same for the sensitivity of forecasted estimation loss in consideration of data partitioning process and the sensitivity of forecasted estimation loss in consideration of data sampling procedure.

Factorized form of the indexing HDMR method for multivariate data modeling

A multivariate data set including a number of scattered nodes with associated function values is given in data modeling problems to construct a rule for the estimation process of unknown function values. To reduce the mathematical and computational complexity of the given problem coming from the multivariance, the given multivariate data set may be partitioned into less-variate data sets with one or two variables. The indexing HDMR method, which is a very recently developed divide-and-conquer method, can be used for this data partitioning process. However, we know that this method works well for the problems with either a dominantly or purely additive nature. To improve the overall performance of the method for different cases, this work offers the factorized form of the Indexing HDMR method. The Factorized Indexing HDMR method uses the Indexing HDMR components to construct the analytical models for the given multivariate problems and impressive numerical results are obtained.

Segmentation of terrestrial laser scanning data using geometry and image information

Terrestrial laser scanning is becoming a standard technology for 3D modeling of complex scenes. Laser scans contain detailed geometric information, but still require interpretation of the data for making it useable for mapping purposes. A fundamental step in the transformation of the data into objects involves their segmentation into consistent units. Such units should follow some predefined rules, and result in salient regions guided by the desire that the individual segments represent object or object-parts within the scene. Nonetheless, due to the scene complexity and the variety of objects, a segmentation using only a single cue does not suffice. Considering the availability of additional data sources such as color images, more information can be integrated in the data partitioning process and ultimately into the reconstruction scheme. We propose segmentation of terrestrial laser scanning data by the integration of range and color content and by using multiple cues. This concept raises questions regarding their mode of integration, and definition of the expected outcome. We show, that while individual segmentation based on given cues have their own limitations, their integration provide a more coherent partitioning that has better potential for further processing. Experiments show that the proposed segmentation methodology yield physically meaningful segments, which surpass those obtained via segmentation of the individual channels.

Multivariate data modelling through Piecewise generalized HDMR method

This work aims to develop a new High Dimensional Model Representation (HDMR) based method which can construct an analytical structure for a given multivariate data modelling problem. Modelling multivariate data through a divide-and-conquer method stands for multivariate data partitioning process in which we deal with a number of less variate data sets instead of a single N dimensional problem. Generalized HDMR is one of these methods used to model a multivariate data set which has a number of scattered nodes with associated function values. However, Generalized HDMR includes a linear equation system with huge number of unknowns and equations to be solved. This equation sometimes has linearly dependent equations in it and this is an undesirable situation. This work offers a new method named Piecewise Generalized HDMR method which bypasses this disadvantage as well as reducing the mathematical complexity and CPU time needed to complete the algorithm of the previous method. Our new method splits the given problem domain into subdomains, applies the Generalized HDMR philosophy to each subdomain and superpositions the information coming from these subdomains. The algorithm of this new method and a number of numerical implementations are given in this paper.

A Matrix Based Indexing HDMR method for multivariate data modelling

Modelling multivariate data of real life problems from engineering, chemistry, physics, mathematics or other related sciences, in which function values are known only at arbitrarily distributed points of the problem domain, is an important and complicated issue since there exist mathematical and computational complexities in the analytical structure construction process coming from the multivariance. The Plain High Dimensional Model Representation (HDMR) method expresses a multivariate problem in terms of less-variate problems. In this work, a Matrix Based Indexing HDMR method is developed to make the Plain HDMR philosophy employable for the multivariate data partitioning process. This new method will have the ability of dealing with less-variate data sets by partitioning the given data set into univariate, bivariate and trivariate data sets. Interpolating these partitioned data sets will construct an approximate analytical structure as the model of the given multivariate data modelling problem.

System-level power-performance tradeoffs for reconfigurable computing

In this paper, we propose a configuration-aware data-partitioning approach for reconfigurable computing. We show how the reconfiguration overhead impacts the data-partitioning process. Moreover, we explore the system-level power-performance tradeoffs available when implementing streaming embedded applications on fine-grained reconfigurable architectures. For a certain group of streaming applications, we show that an efficient hardware/software partitioning algorithm is required when targeting low power. However, if the application objective is performance, then we propose the use of dynamically reconfigurable architectures. We propose a design methodology that adapts the architecture and algorithms to the application requirements. The methodology has been proven to work on a real research platform based on Xilinx devices. Finally, we have applied our methodology and algorithms to the case study of image sharpening, which is required nowadays in digital cameras and mobile phones