Approach For Large Datasets Research Articles

1287-P: Deep Immune Phenotyping of Type 1 Diabetes by Machine Learning

Breakdown of self-tolerance is an important common mechanism in autoimmunity. We use machine learning (ML) to identify common patterns and dissimilarities between type 1 diabetes (T1D) , rheumatoid arthritis (RA) and systemic lupus erythematosus (SLE) based on immune phenotyping. PBMCs were isolated from patients with T1D, RA, SLE, and controls. A FACS approach was applied, and a traditional analysis was compared to a ML method implemented on R and based on self-organizing maps (Fig) . Our pipeline includes unsupervised pre-gating, normalization, FlowSOM clustering, and a statistical model (GLMM) , to check for significant differential abundances of cell populations among the autoimmune conditions. After applying our automated workflow to one T cell panel we could identify 14 new cell clusters present in all the samples. The GLMM test revealed a cluster with a significant difference (p=0.035) and a trending one (p=0.059) on the abundance across the different diseases. In particular, CD4pos T cells expressing high IL-7 receptor (CD127) levels and median amounts of CD15s but low CD25, CD161 and FoxP3 are increased in T1D whereas CD4+CD25++CD15s+FoxP3lowCD161lowCD45RA- cells are increased in SLE. In conclusion, our ML workflow identifies a new subtype of T cells significantly increased in T1D. This unsupervised analysis approach for large datasets enables the discovery of new biomarkers complementing traditional workflows. Disclosure J.Vera-ramos: None. B.Prietl: None. L.Herbsthofer: None. V.Pfeifer: None. P.López-garcía: None. M.Stradner: Consultant; AbbVie Inc., AstraZeneca, Speaker's Bureau; Janssen Pharmaceuticals, Inc. T.Pieber: Advisory Panel; Arecor, Novo Nordisk A/S, Research Support; AstraZeneca, European Union, JDRF, Novo Nordisk Foundation, Sanofi, Speaker's Bureau; Novo Nordisk A/S, Roche Diagnostics. Funding JDRF, LRA, & NMSS Joint RFA (2-SRA-2021-1043-S-B)

A novel Map-Scan-Reduce based density peaks clustering and privacy protection approach for large datasets

Large dataset clustering is a major research issue due to huge processing time. Clustering methods goal at split large dataset into number of groups and each group consists of similar data points. Numerous clustering methods such as partition-based clustering, hierarchical clustering, density-based clustering, spectral clustering, and subspace clustering are presented. These methods failed to produce true (accurate) clusters in less response time. To mitigate the issues of existing methods, in this paper, we propose a novel Map-Scan-Reduce based density peaks (DP) clustering approach to cluster the large datasets. MapReduce is a popular distributed processing framework that has several advantages: it has the ability to resolve any issues that arise with large data volume and it partitioned data in a distributed way, but native MapReduce has certain drawbacks including high communication and computation overhead. In this paper, the Map-Scan-Reduce process is divided into three steps: MAP, SCAN, and REDUCE, which solve the problems of native MapReduce. Users scheduling and data preprocessing is implemented using adaptive neuro fuzzy scheduler and improved version of M-Z-D-S (Max–Min, Z-score, and decimal scaling). Furthermore, clusters privacy is protected using differential privacy method and clusters quality is validated using two matrices such as Silhoutte and Dunn for inter-cluster and intra-cluster validations, respectively. Finally, we conduct experiments to analyze the performance of the proposed work in terms of clustering accuracy, speedup ratio (execution time), and efficiency (precision, adjusted Rand index, and normalized mutual information).

A Kriging-Based Optimization Approach for Large Data Sets Exploiting Points Aggregation Techniques

A kriging-based optimization approach is proposed for problems with large data sets and high dimensionality. Memory usage is maintained via model centering aided by minimizing the impact of information loss on accuracy of new point prediction using points aggregation techniques. The eight-parameter TEAM problem 22 is revisited in the context of computational efficiency and accuracy.

K-mer Content, Correlation, and Position Analysis of Genome DNA Sequences for the Identification of Function and Evolutionary Features.

In genome analysis, k-mer-based comparison methods have become standard tools. However, even though they are able to deliver reliable results, other algorithms seem to work better in some cases. To improve k-mer-based DNA sequence analysis and comparison, we successfully checked whether adding positional resolution is beneficial for finding and/or comparing interesting organizational structures. A simple but efficient algorithm for extracting and saving local k-mer spectra (frequency distribution of k-mers) was developed and used. The results were analyzed by including positional information based on visualizations as genomic maps and by applying basic vector correlation methods. This analysis was concentrated on small word lengths (1 ≤ k ≤ 4) on relatively small viral genomes of Papillomaviridae and Herpesviridae, while also checking its usability for larger sequences, namely human chromosome 2 and the homologous chromosomes (2A, 2B) of a chimpanzee. Using this alignment-free analysis, several regions with specific characteristics in Papillomaviridae and Herpesviridae formerly identified by independent, mostly alignment-based methods, were confirmed. Correlations between the k-mer content and several genes in these genomes have been found, showing similarities between classified and unclassified viruses, which may be potentially useful for further taxonomic research. Furthermore, unknown k-mer correlations in the genomes of Human Herpesviruses (HHVs), which are probably of major biological function, are found and described. Using the chromosomes of a chimpanzee and human that are currently known, identities between the species on every analyzed chromosome were reproduced. This demonstrates the feasibility of our approach for large data sets of complex genomes. Based on these results, we suggest k-mer analysis with positional resolution as a method for closing a gap between the effectiveness of alignment-based methods (like NCBI BLAST) and the high pace of standard k-mer analysis.

Fast Support Vector Machine Classification for Large Data Sets

AbstractNormal support vector machine (SVM) algorithms are not suitable for classification of large data sets because of high training complexity. This paper introduces a novel two-stage SVM classification approach for large data sets. Fast clustering techniques are introduced to select the training data from the original data set for the first stage SVM, and a de-clustering technique is then proposed to recover the training data for the second stage SVM. The proposed two-stage SVM classifier has distinctive advantages on dealing with huge data sets such as those in bioinformatics. Finally, we apply the proposed method on several benchmark problems. Experimental results demonstrate that our approach has good classification accuracy while the training is significantly faster than other SVM classifiers.

A FUZZY FAST CLASSIFICATION FOR SHARE MARKET DATABASE WITH LOWER AND UPPER BOUNDS

In recent years, many researchers focused on the research topic of constructing fuzzy classification system. This study introduces a Fuzzy Fast Classification (FFC) approach for large data sets. It has three phases, in the first phase the large data base is reduced with the entropy by removing the number of attribute. In the second phase an approximate classification is obtained by the mean separation of the data by the total weight, upper and lower approximation line is drawn such that 20% of the record lies near the mean line. In the third phase the classification is refined by using fuzzy logic approach for the 20% of the record since they may fall in any one of the category which need to be carefully examined with the degree of fuzzy value. Experimental results for share market database demonstrate that our approach has good classification accuracy while the training is significantly faster than other SVM classifiers. The proposed classifier has distinctive advantages on dealing with huge data sets.

Fast classification for large data sets via random selection clustering and Support Vector Machines

Support Vector Machines SVMs are high-accuracy classifiers. However, normal SVM algorithms are unsuitable for classification of large data sets because of their training complexity. In this paper, we propose a novel SVM classification approach for large data sets. We first use the random selection to select a small group of training data for the first-stage SVM. Then a de-clustering technique is proposed to recover the training data for the second-stage SVM. This two-stage SVM classifier has distinctive advantages on dealing with huge data sets such as those in bioinformatics. The performance analysis is also given in this paper. Finally, we apply the proposed method on several benchmark problems. Experimental results demonstrate that this approach has good classification accuracy while the training is significantly faster than other SVM classifiers.

The TOKEn project: knowledge synthesis for in silico science

The conduct of investigational studies that involve large-scale data sets presents significant challenges related to the discovery and testing of novel hypotheses capable of supporting in silico discovery science. The use of what are known as Conceptual Knowledge Discovery in Databases (CKDD) methods provides a potential means of scaling hypothesis discovery and testing approaches for large data sets. Such methods enable the high-throughput generation and evaluation of knowledge-anchored relationships between complexes of variables found in targeted data sets. The authors have conducted a multipart model formulation and validation process, focusing on the development of a methodological and technical approach to using CKDD to support hypothesis discovery for in silico science. The model the authors have developed is known as the Translational Ontology-anchored Knowledge Discovery Engine (TOKEn). This model utilizes a specific CKDD approach known as Constructive Induction to identify and prioritize potential hypotheses related to the meaningful semantic relationships between variables found in large-scale and heterogeneous biomedical data sets. The authors have verified and validated TOKEn in the context of a translational research data repository maintained by the NCI-funded Chronic Lymphocytic Leukemia Research Consortium. Such studies have shown that TOKEn is: (1) computationally tractable; and (2) able to generate valid and potentially useful hypotheses concerning relationships between phenotypic and biomolecular variables in that data collection. The TOKEn model represents a potentially useful and systematic approach to knowledge synthesis for in silico discovery science in the context of large-scale and multidimensional research data sets.

Mining and tracking evolving web user trends from large web server logs

AbstractRecently, online organizations became interested in tracking users' behavior on their websites to better understand and satisfy their needs. In response to this need, web usage mining tools were developed to help them use web logs to discover usage patterns or profiles. However, since website usage logs are being continuously generated, in some cases, amounting to a dynamic data stream, most existing tools are still not able to handle their changing nature or growing size. This paper proposes a scalable framework that is capable of tracking the changing nature of user behavior on a website, and represent it in a set of evolving usage profiles. These profiles can offer the best usage representation of user activity at any given time, and they can be used as an input to higher‐level applications such as a web recommendation system. Our specific aim is to make the hierarchical unsupervised niche clustering (HUNC) algorithm more scalable, and to add integrated profile tracking and cluster‐based validation to it. Our experiments on real web log data confirm the validity of our approach for large data sets that previously could not be handled in one shot. Copyright © 2010 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 3: 106‐125, 2010

Support vector machine classification for large data sets via minimum enclosing ball clustering

Support vector machine (SVM) is a powerful technique for data classification. Despite of its good theoretic foundations and high classification accuracy, normal SVM is not suitable for classification of large data sets, because the training complexity of SVM is highly dependent on the size of data set. This paper presents a novel SVM classification approach for large data sets by using minimum enclosing ball clustering. After the training data are partitioned by the proposed clustering method, the centers of the clusters are used for the first time SVM classification. Then we use the clusters whose centers are support vectors or those clusters which have different classes to perform the second time SVM classification. In this stage most data are removed. Several experimental results show that the approach proposed in this paper has good classification accuracy compared with classic SVM while the training is significantly faster than several other SVM classifiers.