High-dimensional Numerical Data Research Articles

Penguins Go Parallel: A Grammar of Graphics Framework for Generalized Parallel Coordinate Plots

Parallel Coordinate Plots (PCP) are a valuable tool for exploratory data analysis of high-dimensional numerical data. The use of PCPs is limited when working with categorical variables or a mix of categorical and continuous variables. In this article, we propose Generalized Parallel Coordinate Plots (GPCP) to extend the ability of PCPs from just numeric variables to dealing seamlessly with a mix of categorical and numeric variables in a single plot. In this process we find that existing solutions for categorical values only, such as hammock plots or parsets become edge cases in the new framework. By focusing on individual observations rather than a marginal frequency we gain additional flexibility. The resulting approach is implemented in the R package ggpcp. Supplementary materials for this article are available online.

An Investigation on Disease Diagnosis and Prediction by Using Modified K-Mean clustering and Combined CNN and ELM Classification Techniques

Data analysis is important for managing a lot of knowledge in the healthcare industry. The older medical study favored prediction over processing and assimilating a massive volume of hospital data. The precise research of health data becomes advantageous for early disease identification and patient treatment as a result of the tremendous knowledge expansion in the biological and healthcare fields. But when there are gaps in the medical data, the accuracy suffers. The use of K-means algorithm is modest and efficient to perform. It is appropriate for processing vast quantities of continuous, high-dimensional numerical data. However, the number of clusters in the given dataset must be predetermined for this technique, and choosing the right K is frequently challenging. The cluster centers chosen in the first phase have an impact on the clustering results as well. To overcome this drawback in k-means to modify the initialization and centroid steps in classification technique with combining (Convolutional neural network) CNN and ELM (extreme learning machine) technique is used. To increase this work, disease risk prediction using repository dataset is proposed. We use different types of machine learning algorithm for predicting disease using structured data. The prediction accuracy of using proposed hybrid model is 99.8% which is more than SVM (support vector machine), KNN (k-nearest neighbors), AB (AdaBoost algorithm) and CKN-CNN (consensus K-nearest neighbor algorithm and convolution neural network).

Data-driven modeling and prediction of non-linearizable dynamics via spectral submanifolds

We develop a methodology to construct low-dimensional predictive models from data sets representing essentially nonlinear (or non-linearizable) dynamical systems with a hyperbolic linear part that are subject to external forcing with finitely many frequencies. Our data-driven, sparse, nonlinear models are obtained as extended normal forms of the reduced dynamics on low-dimensional, attracting spectral submanifolds (SSMs) of the dynamical system. We illustrate the power of data-driven SSM reduction on high-dimensional numerical data sets and experimental measurements involving beam oscillations, vortex shedding and sloshing in a water tank. We find that SSM reduction trained on unforced data also predicts nonlinear response accurately under additional external forcing.

Numerical sensitive data recognition based on hybrid gene expression programming for active distribution networks

Complex and flexible access mode, and frequent data interaction bring about large security risks to data transmission for active distribution networks. How to ensure data security is critical to the safe and stable operation of active distribution networks. Traditional methods, like access control, data encryption, and text filtering based on intelligent algorithms, are difficult to ensure the security of dynamically increased and high-dimensional numerical data transmission in active distribution networks. In this paper, we first propose a rough feature selection algorithm based on the average importance measurement (RFS-AIM) to simplify the complexity of data recognition. Then, we propose a sensitive data recognition function mining algorithm based on RFS-AIM and improved gene expression programming (SDR-IGEP) where population update operation is constructed by chromosome similarity based on the Jaccard coefficient. The operation avoids local convergence of the gene express programming by increasing individual diversity in the new population. Finally, we present a new incremental mining algorithm for a sensitive data recognition function based on global function fitting (ISDR-GFF) by using a grain granulation model for incremental datasets. The experimental results on IEEE benchmark datasets and real datasets show that the algorithms proposed in this paper outperform the state-of-the-art algorithms in terms of the average running time, precision, recall, F1 index, accuracy, specificity and speedup on all experimental datasets.

Digital staining through the application of deep neural networks to multi-modal multi-photon microscopy.

Deep neural networks have been used to map multi-modal, multi-photon microscopy measurements of a label-free tissue sample to its corresponding histologically stained brightfield microscope colour image. It is shown that the extra structural and functional contrasts provided by using two source modes, namely two-photon excitation microscopy and fluorescence lifetime imaging, result in a more faithful reconstruction of the target haematoxylin and eosin stained mode. This modal mapping procedure can aid histopathologists, since it provides access to unobserved imaging modalities, and translates the high-dimensional numerical data generated by multi-modal, multi-photon microscopy into traditionally accepted visual forms. Furthermore, by combining the strengths of traditional chemical staining and modern multi-photon microscopy techniques, modal mapping enables label-free, non-invasive studies of in vivo tissue samples or intravital microscopic imaging inside living animals. The results show that modal co-registration and the inclusion of spatial variations increase the visual accuracy of the mapped results.

Two improved k-means algorithms

K-means algorithm is the most commonly used simple clustering method. For a large number of high dimensional numerical data, it provides an efficient method for classifying similar data into the same cluster. In this study, a tri-level k-means algorithm and a bi-layer k-means algorithm are proposed. The k-means algorithm is vulnerable to outliers and noisy data, and also susceptible to initial cluster centers. The tri-level k-means algorithm can overcome these drawbacks. While the data in a dataset S are often changed, after a period of time the trained cluster centers cannot precisely describe the data in each cluster. The cluster centers hence need to be updated. In this paper, an online machine learning based tri-level k-means algorithm is also provided to solve this problem. When the data in a cluster are significantly different, a cluster center cannot alone precisely describe each datum in the cluster. Noisy data, outliers, and data with quite different values in the same cluster may decrease the performance of pattern matching systems. The bi-layer k-means algorithm can deal with the above problems. Meanwhile, a genetic-based algorithm is provided to derive the fittest parameters used in the tri-level and bi-layer k-means algorithms. Experimental results demonstrate that both algorithms can provide much better accuracy of classification than the traditional k-means algorithm.

Shinyheatmap: Ultra fast low memory heatmap web interface for big data genomics.

BackgroundTranscriptomics, metabolomics, metagenomics, and other various next-generation sequencing (-omics) fields are known for their production of large datasets, especially across single-cell sequencing studies. Visualizing such big data has posed technical challenges in biology, both in terms of available computational resources as well as programming acumen. Since heatmaps are used to depict high-dimensional numerical data as a colored grid of cells, efficiency and speed have often proven to be critical considerations in the process of successfully converting data into graphics. For example, rendering interactive heatmaps from large input datasets (e.g., 100k+ rows) has been computationally infeasible on both desktop computers and web browsers. In addition to memory requirements, programming skills and knowledge have frequently been barriers-to-entry for creating highly customizable heatmaps.ResultsWe propose shinyheatmap: an advanced user-friendly heatmap software suite capable of efficiently creating highly customizable static and interactive biological heatmaps in a web browser. shinyheatmap is a low memory footprint program, making it particularly well-suited for the interactive visualization of extremely large datasets that cannot typically be computed in-memory due to size restrictions. Also, shinyheatmap features a built-in high performance web plug-in, fastheatmap, for rapidly plotting interactive heatmaps of datasets as large as 105—107 rows within seconds, effectively shattering previous performance benchmarks of heatmap rendering speed.Conclusionsshinyheatmap is hosted online as a freely available web server with an intuitive graphical user interface: http://shinyheatmap.com. The methods are implemented in R, and are available as part of the shinyheatmap project at: https://github.com/Bohdan-Khomtchouk/shinyheatmap. Users can access fastheatmap directly from within the shinyheatmap web interface, and all source code has been made publicly available on Github: https://github.com/Bohdan-Khomtchouk/fastheatmap.

Eureka!: an interactive and visual knowledge discovery tool

Visualization techniques may guide the data mining process since they provide effective support for data partitioning and visual inspection of results, especially when high dimensional data sets are considered. In this paper we describe Eureka!, an interactive, visual knowledge discovery tool for analyzing high dimensional numerical data sets. The tool combines a visual clustering method, to hypothesize meaningful structures in the data, and a classification machine learning algorithm, to validate the hypothesized structures. A two-dimensional representation of the available data allows users to partition the search space by choosing shape or density according to criteria they deem optimal. A partition can be composed by regions populated according to some arbitrary form, not necessarily spherical. The accuracy of clustering results can be validated by using different techniques (e.g. a decision tree classifier) included in the mining tool.