Underlying Data Structure Research Articles

Fuzzy Clustering With Knowledge Extraction and Granulation

Knowledge-based clustering algorithms can improve traditional clustering models by introducing domain knowledge to identify the underlying data structure. While there have been several approaches to clustering with the guidance of knowledge tidbits, most of them mainly focus on numeric knowledge without considering the uncertain nature of information. To capture the uncertainty of information, pure numeric knowledge tidbits are expanded to knowledge granules in this article. Then, two questions arise: how to obtain granular knowledge and how to use those knowledge granules in clustering. To the end, a novel knowledge extraction and granulation (KEG) method and a granular knowledge-based fuzzy clustering model are proposed in this study. First, inspired by the concept of natural neighbors, an automatic KEG is developed. In KEG, high-density points are filtered from the dataset and then merged with their natural neighbors to form several dense areas, i.e., granular knowledge. Furthermore, the granular knowledge expressed by interval or triangular numbers is leveraged into the clustering algorithm, which is the framework of fuzzy clustering with granular knowledge. To concretize this model into clustering algorithms, the classical fuzzy C-Means clustering algorithm has been selected to incorporate the granular knowledge produced by KEG. Then, the corresponding fuzzy C-Means clustering with interval knowledge granules (IKG-FCM) and triangular knowledge granules (TKG-FCM) are proposed. Experiments on synthetic and real-world datasets demonstrate that IKG-FCM and TKG-FCM always achieve better clustering performance with less time cost, especially on imbalanced data, compared with state-of-the-art algorithms.

Data-driven comparison of federated learning and model personalization for electric load forecasting

Residential short-term electric load forecasting is essential in modern decentralized power systems. Load forecasting methods mostly rely on neural networks and require access to private and sensitive electric load data for model training. Conventional neural network training aggregates all data on a centralized server to train one global model. However, the aggregation of user data introduces security and data privacy risks. In contrast, this study investigates the modern neural network training methods of federated learning and model personalization as potential solutions to security and data privacy problems. Within an extensive simulation approach, the investigated methods are compared to the conventional centralized method and a pre-trained baseline predictor to compare their respective performances. This study identifies that the underlying data structure of electric load data has a significant influence on the loss of a model. We therefore conclude that a comparison of loss distributions will in fact be considered a comparison of data structures, rather than a comparison of the model performance. As an alternative method of comparison of loss values, this study develops the ”differential comparison”. The method allows for the isolated comparison of model loss differences by only comparing the losses of two models generated by the same data sample to build a distribution of differences. The differential comparison method was then used to identify model personalization as the best performing model training method for load forecasting among all analyzed methods, with a superior performance in 59.1 % of all cases.

Analysis assaulting pattern for the security problem monitoring in 5G‐enabled sensor network systems with big data environment using artificial intelligence/machine learning

AbstractThe 5G‐enabled sensor network systems make it possible to connect cyber and real ‘things’ in many ways. Even so, the flow of data between 5G‐enabled sensor devices brings big data environment problems, such as huge amounts of data, duplicated data, a lack of scalability etc. Some of these problems are as follows. It is hard to keep an eye on 5G‐enabled sensor systems in a ‘big data’ environment. Cyberattacks that could put the safety of the sensor network systems at risk are hard to find, which make the situation even more complicated. The security challenges of 5G‐enabled Sensor Network Systems are studied and analyzed due to some constraints associated with the sensor nodes. The proposed advanced algorithm for securing the 5G‐enabled sensor systems is a Multidimensional big data environment using artificial intelligence/machine learning (AI/ML). Using a structure that depends on both geographical and temporal data, an improved clear point selection operation may get important information from multidimensional time series data that is spread across a wide range of sensor nodes. Therefore, the actions of the 5G‐enabled sensor network can be shown accurately and a complete model of its underlying data structure is built to analysis attacking, pattern on 5G‐enabled Sensor Network Systems using the AI/ML Algorithm.

XDLX: A Memory-Efficient Solution for Backtracking Applications in Big Data Environment using XOR-based Dancing Links

Interactive and Backtracking applications often require undoing certain recent operations and updates made to the underlying data structures. The concept of dancing links has made such reverting operations easier and efficient by repeatedly performing unlinking and re-linking of pointers in complex data structures involving circular multiply linked lists. This paper extends the idea of dancing links to XOR linked lists, the memory efficient counterpart of doubly linked lists to develop XDLX, a more space efficient algorithm than DLX to solve exact cover problems without compromising the timing efficiency. Owing to the NP-Complete nature of the exact cover problem, any NP-complete problem can be reduced to it and solved using the proposed memory-efficient dancing links based algorithm, XDLX. The algorithm can be effectively used to solve any backtracking application and will prove to be a significant contribution towards the programming of memory-constrained environments such as embedded systems.

An Improved SVM with Earth Mover’s Distance Regularization and Its Application in Pattern Recognition

A support vector machine (SVM) aims to achieve an optimal hyperplane with a maximum interclass margin and has been widely utilized in pattern recognition. Traditionally, a SVM mainly considers the separability of boundary points (i.e., support vectors), while the underlying data structure information is commonly ignored. In this paper, an improved support vector machine with earth mover’s distance (EMD-SVM) is proposed. It can be regarded as an improved generalization of the standard SVM, and can automatically learn the distribution between the classes. To validate its performance, we discuss the necessity of the structural information of EMD-SVM in the linear and nonlinear cases, respectively. Experimental validation was designed and conducted in different application fields, which have shown its superior and robust performance.

Self-taught Multi-view Spectral Clustering

By integrating multiple views, i.e., multi-view learning (ML), we can discover the underlying data structures so that the performance of learning tasks can improve. As a basic and important branch of ML, multi-view clustering has achieved great success recently in pattern recognition and machine learning communities. Most existing multi-view spectral clustering methods heavily adopt the relax-and-discretize strategy to obtain discrete cluster labels (clustering results), i.e., using predefined similarity graphs to learn a consensus Laplacian embedding shared by all views for K-means clustering. However, the above clustering strategy may significantly affect clustering performance since there is information loss between independent steps. In this paper, we establish a novel Self-taught Multi-view Spectral Clustering (SMSC) framework to address the above issue. As the main contributions of this paper, we provide two versions of SMSC based on convex combination and centroid graph fusion schemes. Specifically, a self-taught mechanism is introduced in SMSC, which can effectively feedback the manifold structure induced by Laplacian embedding and the cluster information hidden in the discrete indicator matrix to learn an optimal consensus similarity graph for graph partitioning. The effectiveness of the proposed methods has been evaluated on real-world multi-view datasets, and experimental results show that our methods outperform other state-of-the-art baselines.

Using multiple outcomes in intervention studies: improving power while controlling type I errors.

Background The CONSORT guidelines for clinical trials recommend using a single primary outcome, to guard against excess false positive findings when multiple measures are considered. However, statistical power can be increased while controlling the familywise error rate if multiple outcomes are included. The MEff statistic is well-suited to this purpose, but is not well-known outside genetics. Methods Data were simulated for an intervention study, with a given sample size (N), effect size (E) and correlation matrix for a suite of outcomes ( R). Using the variance of eigenvalues from the correlation matrix, we compute MEff, the effective number of variables that the alpha level should be divided by to control the familywise error rate. Various scenarios are simulated to consider how MEff is affected by the pattern of pairwise correlations within a set of outcomes. The power of the MEff approach is compared to Bonferroni correction, and a principal component analysis (PCA). Results In many situations, power can be increased by inclusion of multiple outcomes. Differences in power between MEff and Bonferroni correction are small if intercorrelations between outcomes are low, but the advantage of MEff is more evident as intercorrelations increase. PCA is superior in cases where the impact on outcomes is fairly uniform, but MEff is applicable when intervention effects are inconsistent across measures. Conclusions The optimal method for correcting for multiple testing depends on the underlying data structure, with PCA being superior if outcomes are all indicators of a common underlying factor. Both Bonferroni correction and MEff can be applied post hoc to evaluate published intervention studies, with MEff being superior when outcomes are moderately or highly correlated. A lookup table is provided to give alpha levels for use with Meff for cases where the correlation between outcome measures can be estimated.

HeLayers: A Tile Tensors Framework for Large Neural Networks on Encrypted Data

Privacy-preserving solutions enable companies to offload confidential data to third-party services while fulfilling their government regulations. To accomplish this, they leverage various cryptographic techniques such as Homomorphic Encryption (HE), which allows performing computation on encrypted data. Most HE schemes work in a SIMD fashion, and the data packing method can dramatically affect the running time and memory costs. Finding a packing method that leads to an optimal performant implementation is a hard task. We present a simple and intuitive framework that abstracts the packing decision for the user. We explain its underlying data structures and optimizer, and propose a novel algorithm for performing 2D convolution operations. We used this framework to implement an inference operation over an encrypted HE-friendly AlexNet neural network with large inputs, which runs in around five minutes, several orders of magnitude faster than other state-of-the-art non-interactive HE solutions.

π-means: Granular Approach towards Interactive Data Exploration

In this paper, we examine the possibility of employing the idea of progressive-inductive (π) aggregation in the k-means algorithm. We base our work on the interactive visualization framework called Skydive which is a tightly coupled system that combines data aggregation and visual aggregation for smooth user interaction. It is based on an aggregate pyramid data structure (π-cube), allowing a visualization client to quickly filter the data. Skydive was designed for interactive data visualization in terms of expressiveness and efficiency. Therefore, the underlying data structure enabled access to data on desired levels of granularity. In this paper, we utilize this concept to speed up the process of clustering proposing π-means - a modified version of the k-means algorithm. This will allow us to explore further possible applications of π-cube towards interactive data exploration.

Autoweighted Multiview Feature Selection With Graph Optimization.

In this article, we focus on the unsupervised multiview feature selection, which tries to handle high-dimensional data in the field of multiview learning. Although some graph-based methods have achieved satisfactory performance, they ignore the underlying data structure across different views. Besides, their predefined Laplacian graphs are sensitive to the noises in the original data space and fail to obtain the optimal neighbor assignment. To address the above problems, we propose a novel unsupervised multiview feature selection model based on graph learning, and the contributions are three-fold: 1) during the feature selection procedure, the consensus similarity graph shared by different views is learned. Therefore, the proposed model can reveal the data relationship from the feature subset; 2) a reasonable rank constraint is added to optimize the similarity matrix to obtain more accurate information; and 3) an autoweighted framework is presented to assign view weights adaptively, and an effective alternative iterative algorithm is proposed to optimize the problem. Experiments on various datasets demonstrate the superiority of the proposed method compared to the state-of-the-art methods.

Single‐linkage clustering of dynamic data

SummaryThe surge in data sizes in fluid processing applications necessitates partitioning the data into clusters and studying their representatives instead of studying each voxel data point. In addition, the dynamic nature of these data poses further challenges. Under such circumstances, it becomes essential to develop an approach that can handle the delta data with minimal updates to the underlying data structure, without processing the complete data from scratch on every update. However, this poses synchronization challenges in parallelization. In this article, we propose SLCoDD (single‐linkage clustering of dynamic data), a geometric distance based dynamic clustering and its multi‐core parallelization using OpenMP. To improve efficiency, SLCoDD exploits geometric properties of the bounding squares. We illustrate trade‐offs in various ways of performing point additions to clusters, point deletions, and their batched versions. Using a suite of large inputs, we demonstrate the effectiveness of SLCoDD. SLCoDD's fully dynamic version achieves a substantial geomean speedup of over the static parallel version and of over the dynamic sequential version.

Discriminative and Geometry-Preserving Adaptive Graph Embedding for dimensionality reduction

Learning graph embeddings for high-dimensional data is an important technology for dimensionality reduction. The learning process is expected to preserve the discriminative and geometric information of high-dimensional data in a new low-dimensional subspace via either manual or automatic graph construction. Although both manual and automatic graph constructions can capture the geometry and discrimination of data to a certain degree, they working alone cannot fully explore the underlying data structure. To learn and preserve more discriminative and geometric information of the high-dimensional data in the low-dimensional subspace as much as possible, we develop a novel Discriminative and Geometry-Preserving Adaptive Graph Embedding (DGPAGE). It systematically integrates manual and adaptive graph constructions in one unified graph embedding framework, which is able to effectively inject the essential information of data involved in predefined graphs into the learning of an adaptive graph, in order to achieve both adaptability and specificity of data. Learning the adaptive graph jointly with the optimized projections, DGPAGE can generate an embedded subspace that has better pattern discrimination for image classification. Results derived from extensive experiments on image data sets have shown that DGPAGE outperforms the state-of-the-art graph-based dimensionality reduction methods. The ablation studies show that it is beneficial to have an integrated framework, like DGPAGE, that brings together the advantages of manual/adaptive graph construction.

Monadic Expressions and their Derivatives

We propose another interpretation of well-known derivatives computations from regular expressions, due to Brzozowski, Antimirov or Lombardy and Sakarovitch, in order to abstract the underlying data structures (e.g. sets or linear combinations) using the notion of monad. As an example of this generalization advantage, we introduce a new derivation technique based on the graded module monad. We also extend operators defining expressions to any n-ary functions over value sets, such as classical operations (like negation or intersection for Boolean weights) or more exotic ones (like algebraic mean for rational weights). Moreover, we present how to compute a (non-necessarily finite) automaton from such an extended expression, using the Colcombet and Petrisan categorical definition of automata. These category theory concepts allow us to perform this construction in a unified way, whatever the underlying monad. Finally, to illustrate our work, we present a Haskell implementation of these notions using advanced techniques of functional programming, and we provide a web interface to manipulate concrete examples.

Clustering ensemble based on approximate accuracy of the equivalence granularity

Clustering ensemble deals with the inconsistent parts of base clusterings while maintaining consistency in the clustering results. The cluster core samples generally have more consistent neighbor relationship in base clusterings than the cluster edge samples, and these two types of samples have different contributions to the determination of the underlying data structure. In addition, in clustering ensemble, some samples that are equivalent in base clusterings can be merged into equivalence granularity to reduce the size of input data. Based on these ideas and combined with rough set theory, we propose a novel clustering ensemble algorithm based on the approximate accuracy of equivalence granularity. We initially define the clustering equivalence relation, extract the consistent results of base clusterings based on this relation, and upgrade the clustering ensemble from the sample level to the equivalence granularity level. Afterward, we use the approximate accuracy of the rough set to quantify the contribution of equivalence granularity in discovering the underlying data structure, divide the equivalence granularities into core and edge equivalence granularities, and implement different clustering processing strategies. The visual experiments on four synthetic data sets and comparison experiments with 14 state-of-the-art clustering ensemble algorithms on 21 data sets respectively prove the rationality and excellent performance of our proposed algorithm.

Improving EEG Decoding via Clustering-Based Multitask Feature Learning.

Accurate electroencephalogram (EEG) pattern decoding for specific mental tasks is one of the key steps for the development of brain-computer interface (BCI), which is quite challenging due to the considerably low signal-to-noise ratio of EEG collected at the brain scalp. Machine learning provides a promising technique to optimize EEG patterns toward better decoding accuracy. However, existing algorithms do not effectively explore the underlying data structure capturing the true EEG sample distribution and, hence, can only yield a suboptimal decoding accuracy. To uncover the intrinsic distribution structure of EEG data, we propose a clustering-based multitask feature learning algorithm for improved EEG pattern decoding. Specifically, we perform affinity propagation-based clustering to explore the subclasses (i.e., clusters) in each of the original classes and then assign each subclass a unique label based on a one-versus-all encoding strategy. With the encoded label matrix, we devise a novel multitask learning algorithm by exploiting the subclass relationship to jointly optimize the EEG pattern features from the uncovered subclasses. We then train a linear support vector machine with the optimized features for EEG pattern decoding. Extensive experimental studies are conducted on three EEG data sets to validate the effectiveness of our algorithm in comparison with other state-of-the-art approaches. The improved experimental results demonstrate the outstanding superiority of our algorithm, suggesting its prominent performance for EEG pattern decoding in BCI applications.

TidyMass an object-oriented reproducible analysis framework for LC–MS data

Reproducibility, traceability, and transparency have been long-standing issues for metabolomics data analysis. Multiple tools have been developed, but limitations still exist. Here, we present the tidyMass project (https://www.tidymass.org/), a comprehensive R-based computational framework that can achieve the traceable, shareable, and reproducible workflow needs of data processing and analysis for LC-MS-based untargeted metabolomics. TidyMass is an ecosystem of R packages that share an underlying design philosophy, grammar, and data structure, which provides a comprehensive, reproducible, and object-oriented computational framework. The modular architecture makes tidyMass a highly flexible and extensible tool, which other users can improve and integrate with other tools to customize their own pipeline.

Towards an Ontology-Based Phenotypic Query Model

Clinical research based on data from patient or study data management systems plays an important role in transferring basic findings into the daily practices of physicians. To support study recruitment, diagnostic processes, and risk factor evaluation, search queries for such management systems can be used. Typically, the query syntax as well as the underlying data structure vary greatly between different data management systems. This makes it difficult for domain experts (e.g., clinicians) to build and execute search queries. In this work, the Core Ontology of Phenotypes is used as a general model for phenotypic knowledge. This knowledge is required to create search queries that determine and classify individuals (e.g., patients or study participants) whose morphology, function, behaviour, or biochemical and physiological properties meet specific phenotype classes. A specific model describing a set of particular phenotype classes is called a Phenotype Specification Ontology. Such an ontology can be automatically converted to search queries on data management systems. The methods described have already been used successfully in several projects. Using ontologies to model phenotypic knowledge on patient or study data management systems is a viable approach. It allows clinicians to model from a domain perspective without knowing the actual data structure or query language.

MULTI-FREQUENCY POLINSAR DATA ARE ADVANTAGEOUS FOR LAND COVER CLASSIFICATION – A VISUAL AND QUANTITATIVE ANALYSIS

Abstract. This paper investigates the enhanced potential of using multi-frequency PolInSAR data for land cover classification. In order to enable a descriptive analysis that goes beyond the mere comparison of classification accuracies, a two-step classification process is applied. First, polarimetric and interferometric features are extracted and projected into a 3-dimensional feature space by using the supervised dimension reduction algorithm Uniform Manifold Approximation and Projection (UMAP). Subsequently, based on the expressive 3-dimensional representation a simple yet sufficient k-nearest neighbors (KNN) classifier is applied to assign a land cover class to each pixel. In this way, besides the simplified classification, the visualization of the underlying data structure is possible and contributes to a better explanation and analysis of classification results. The data analyzed in this way are airborne L- and S-band PolInSAR data acquired by the F-SAR system. The visual analysis of reduced feature spaces as well as the quantitative analysis of classification results reveal the benefits of combining both frequencies with regard to class separability.

Improving Pulse Shape Discrimination in Organic Scintillation Detectors by Understanding Underlying Data Structure

Various machine learning techniques have been implemented to assist in neutron-gamma discrimination with great success compared to traditional methods. Despite this, the fundamental structure of a pulse shape as it relates to machine learning has not yet been explored in detail, and the optimal number of pulse vector features needed for training is still unknown. In this study, support vector machines (SVMs) using linear, radial basis, and exponential kernel functions are fitted on data of two different forms: waveforms that partially cover the original pulses and principal components extracted from those pulses. The described methods correctly classified 98.02% for neutrons and 97.84% for gamma rays. The efficiency of the SVM was improved by extracting principal components from the waveforms. That is, fewer features were needed to discriminate between neutrons and gamma rays without negatively impacting the classification accuracy. This study also shows that utilizing a nonlinear kernel significantly reduces the number of features required to reach high classification accuracy. SVMs that did this could make accurate classifications 97% of the time with data that had fewer than 50 features.

Empowering multi-class medical data classification by Group-of-Single-Class-predictors and transfer optimization: Cases of structured dataset by machine learning and radiological images by deep learning

In the medical domain, data are often collected over time, evolving from simple to refined categories. The data and the underlying structures of the medical data as to how they have grown to today’s complexity can be decomposed into crude forms when data collection starts. For instance, the cancer dataset is labeled either benign or malignant at its simplest or perhaps the earliest form. As medical knowledge advances and/or more data become available, the dataset progresses from binary class to multi-class, having more labels of sub-categories of the disease added. In machine learning, inducing a multi-class model requires more computational power. Model optimization is enforced over the multi-class models for the highest possible accuracy, which of course, is necessary for life-and-death decision making. This model optimization task consumes an extremely long model training time. In this paper, a novel strategy called Group-of-Single-Class prediction (GOSC) coupled with majority voting and model transfer is proposed for achieving maximum accuracy by using only a fraction of the model training time. The main advantage is the ability to achieve an optimized multi-class classification model that has the highest possible accuracy near to the absolute maximum, while the training time could be saved by up to 70%. Experiments on machine learning over liver dataset classification and deep learning over COVID19 lung CT images were tested. Preliminary results suggest the feasibility of this new approach.