Increase In Complexity Of Data Research Articles

An Event Frequency‐Inverse Session Frequency Based Weighting Mechanism for Data Embedding Methodologies

ABSTRACTContextWith the rapid increase in data complexity and volume, analyzing and understanding large‐scale datasets has become a critical challenge, especially in domains like e‐commerce. This paper addresses this challenge by introducing a novel business process that utilizes an event frequency‐inverse session frequency (EF‐ISF) based weighting mechanism for embedding user navigational behavior in e‐commerce websites.ObjectiveThe primary research problem tackled in this study is the detection of patterns and distributions within user behavior data, which is often complicated by dynamic page content and session sparsity. We aim to enhance the interpretability and accuracy of user behavior analysis by integrating EF‐ISF with various embedding techniques, including Word2Vec, Node2Vec, DeepWalk, Struc2Vec, and LSTM Autoencoder.MethodologyOur approach is particularly effective in datasets where page refreshes lead to event repetitions, where sequential events with temporal dependencies are prevalent, and where event distributions across sessions are sparse. By applying this methodology across multiple e‐commerce datasets, we observe consistent performance in clustering quality, successfully detecting underlying patterns and groups within the data.ResultsA prototype implementation demonstrates the practical utility of the proposed method, with evaluation metrics confirming that EF‐ISF weighting not only enhances the embeddings but also facilitates a more nuanced and significant segmentation of users' browsing behaviors.ConclusionThese findings provide a valuable contribution to the field of e‐commerce data analytics, offering businesses a refined approach to optimizing user experience and engagement through a deeper understanding of user behavior patterns. This methodology can significantly improve the quality of insights derived from user behavior data, ultimately leading to better decision‐making and enhanced user satisfaction.

STATE: A Clustering Algorithm Focusing on Edges Instead of Centers

With the expansion of data scale and the increase in data complexity, it is particularly important to accurately identify clusters and efficiently save clustering results. To address this, we propose a novel clustering algorithm, Shape clustering based on data field (STATE), which can quickly identify clusters of arbitrary shapes and greatly reduce the storage space of clustering results in any datasets without reducing the accuracy. STATE mainly focuses on finding the edges of clusters and directions of edges instead of clustering centers through the data field. The results of STATE are presented as the edges of clusters without data objects inside clusters and without noise. Extensive experiments show that STATE can recognize complex data distribution in noisy environments without discrimination and greatly save the storage space of clustering results. When it is applied in a real-world scene, facial feature extraction, STATE can recognize eyes, nose, mouth, eyebrows and facial contours automatically without calibrating key features or training. Using the extracted facial features, we achieve facial recognition with high accuracy.

Rolling Element Bearing Fault Diagnosis Based on Deep Belief Network and Principal Component Analysis

Rolling element bearings are critical components in industrial rotating machines. Faults and failures of bearings can cause degradation of machine performance or even a catastrophe. Bearing fault diagnosis is therefore essential and significant to safe and reliable operation of systems. For bearing condition monitoring, acoustic emission (AE) signals attract more and more attention due to its advantages on sensitivity over the extensively used vibration signal. In bearing fault diagnosis and prognosis, feature extraction is a critical and tough work, which always involves complex signal processing and computation. Moreover, features greatly rely on the characteristics, operating conditions, and type of data. With consideration of changes in operating conditions and increase of data complexity, traditional diagnosis approaches are insufficient in feature extraction and fault diagnosis. To address this problem, this paper proposes a Deep Belief Network (DBN) and Principal Component Analysis (PCA) based fault diagnosis approach using AE signal. This proposed approach combines the advantages of deep learning and statistical analysis, DBN automatically extracts features from AE signal, PCA is applied to dimensionality reduction. Different bearing fault modes are identified by least squares support vector machine (LS-SVM) using the extracted features. An experimental case is conducted with a tapered roller bearing to verify the proposed approach. Experimental results demonstrate that the proposed approach has excellent feature extraction ability and high fault classification accuracy.

Hierarchical Request-Size-Aware Flash Translation Layer Based on Page-Level Mapping

Owing to the increasing Internet population, there has been an explosion in the amount of digital data generated and also an increase in data complexity. This trend is called big data paradigm. As the Internet of Things (IoT) takes center stage, the growth of data will continue to increase. Therefore, the demand for mass storage devices that have high access speed is increasing. Industry has been paying attention to flash memories that can process large amounts of data at high speed. It will be a good alternative for storing and processing ever-increasing amounts of data because of low power consumption, high shock resistance, portability and fast access speed. However, the write speed is about 10–20 times slower than the read speed in flash memory. In addition, write operations are not allowed to be performed with in-place updates. Garbage collection mechanism is proposed in order to solve the problem incurred by the not-in-place update property of write operations. However, garbage collection mechanism unavoidably causes overhead of additional internal operations, which leads to performance degradation. In this paper, to prevent performance degradation caused by garbage collection, we propose a request-size-aware flash translation layer (RSaFTL) and a hierarchical request-size-aware flash translation layer (HiRSaFTL). They are designed based on page-level address translation. In RSaFTL and HiRSaFTL, page-sized data with high temporal locality cluster into a special area called active blocks by exploiting the property of realistic traces. As a result of the experiments, RSaFTL and HiRSaFTL reduce the number of pages migrated during garbage collections by up to 17.9% and 21.3%, respectively, compared with pure page-level flash transition layer.

Multidimensional Contrast Limited Adaptive Histogram Equalization

Contrast enhancement is an important preprocessing technique for improving the performance of downstream tasks in image processing and computer vision. Among the existing approaches based on nonlinear histogram transformations, contrast limited adaptive histogram equalization (CLAHE) is a popular choice for dealing with 2D images obtained in natural and scientific settings. The recent hardware upgrade in data acquisition systems results in significant increase in data complexity, including their sizes and dimensions. Measurements of densely sampled data higher than three dimensions, usually composed of 3D data as a function of external parameters, are becoming commonplace in various applications in the natural sciences and engineering. The initial understanding of these complex multidimensional datasets often requires human intervention through visual examination, which may be hampered by the varying levels of contrast permeating through the dimensions. We show both qualitatively and quantitatively that using our multidimensional extension of CLAHE (MCLAHE) simultaneously on all dimensions of the datasets allows better visualization and discernment of multidimensional image features, as demonstrated using cases from 4D photoemission spectroscopy and fluorescence microscopy. Our implementation of multidimensional CLAHE in Tensorflow is publicly accessible and supports parallelization with multiple CPUs and various other hardware accelerators, including GPUs.

A discrete dynamics approach to sparse calculation and applied in ontology science

ABSTRACT In the era of big data, with the increase of data processing information and the increase of data complexity, higher requirements are put on the tools and algorithms of data processing. As a tool for structured information representation, ontology has been used in engineering fields such as chemistry, biology, pharmacy, and materials. As a dynamic structure, the increasing concepts contributes to a gradual increase of a single ontology. In order to solve the problem of computational complexity decreasing in the procedure of similarity calculating, the techniques of dimensionality reduction and sparse computing are applied to ontology learning. This article presents discrete dynamics approach showing several tricks on applying the sparse computing method to ontology learning, and verify its efficiency through experiments.

A Survey on Artificial Intelligence Techniques to Prevent Cyber Crime

Abstract-- With increase in complexity of data, security, it is difficult for the individuals to prevent the offence. Thus, by using any automation or software it’s not possible by only using huge fixed algorithms to overcome this. Thus, we need to look for something which is robust and feasible enough. Hence AI plays an epitome role to defense such violations. In this paper we basically look how human reasoning along with AI can be applied to uplift cyber security.

Applications of clustering model to bimodal distributions for dielectric breakdown

In this work, the authors report the extensive time-to-breakdown (TBD) data collected from back end of line/middle of the line and metal insulator metal capacitor dielectrics that exhibit not only non-Poisson area scaling but also multiple modal (or bimodal) breakdown characteristics. They develop a new bimodal modeling approach of the distributed competition process in conjunction with the time-dependent clustering model to extract the breakdown parameters and the characteristic breakdown time and slope (t63 and β) from this seemingly intractable breakdown data. While the number of parameters increases as a result of an increase in data complexity, these extracted parameters are consistent with Poisson area-scaling of fundamental weakest-link characteristics, and can thus be used for reliability projection. As a result, our modeling approach involving clustering model provides an important solution for technology qualification and dielectric-integrity assessment.

Abstract 3976: NetGestalt: A web application for the integration of multidimensional omics data within a network context

Abstract Advancements in high-throughput omics technologies are revolutionizing the field of cancer studies. As a prime example, The Cancer Genome Atlas (TCGA) project will generate multiple types of genomic data from 500 cases of human cancer for each of the 25 selected tumor types by 2014. Until computational tools are available for biologists and clinicians to independently interpret the vast amount of interconnected data, the potential of this type of large collaborative projects will not be fully realized. To fill the gap between data generation and investigators’ ability to interpret the data, we have developed NetGestalt, a web-based application that enables the integration of multidimensional omics data within the context of a protein interaction network. Because molecular alterations at DNA, RNA, and protein levels exert their effects primarily through changing the activity of proteins and their participating networks, protein-protein interaction network has become a powerful model for the visualization and integration of different types of molecular data. However, the standard graph-based network visualization becomes inadequate as network size and data complexity increase. We address this challenge through exploiting the inherent hierarchical architecture of the protein interaction network. By using only the horizontal dimension of a webpage to layout genes according to the hierarchical network architecture, it allows users to simultaneously compare and correlate information from experimental data, network modules, and existing knowledge rendered as tracks along the vertical dimension of the webpage, similar to the widely used genome browsers. However, without constraining the system to genomic sequence-based coordinates, NetGestalt is able to reveal functional relationship between different genes as encoded in the network. We also employed efficient software architecture to enable fast track rendering process and smooth navigation between different resolution scales from individual genes to the whole network. The potential of NetGestalt was demonstrated using the recently published TCGA ovarian cancer data with multiple types of genomic measurements on around 500 tumor samples and corresponding normal controls. In summary, we have developed NetGestalt, a novel data integration framework that allows simultaneous presentation of large scale experimental and annotation data from various sources in the context of a biological network to facilitate data visualization, analysis, interpretation and hypothesis generation. A NetGestalt-based TCGA data browser can facilitate biologists and clinicians to translate the vast amount of data into novel biological discoveries and better cancer therapeutics. Citation Format: {Authors}. {Abstract title} [abstract]. In: Proceedings of the 103rd Annual Meeting of the American Association for Cancer Research; 2012 Mar 31-Apr 4; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2012;72(8 Suppl):Abstract nr 3976. doi:1538-7445.AM2012-3976

Reconstructing embryonic development

Novel approaches to bio-imaging and automated computational image processing allow the design of truly quantitative studies in developmental biology. Cell behavior, cell fate decisions, cell interactions during tissue morphogenesis, and gene expression dynamics can be analyzed in vivo for entire complex organisms and throughout embryonic development. We review state-of-the-art technology for live imaging, focusing on fluorescence light microscopy techniques for system-level investigations of animal development, and discuss computational approaches to image segmentation, cell tracking, automated data annotation, and biophysical modeling. We argue that the substantial increase in data complexity and size requires sophisticated new strategies to data analysis to exploit the enormous potential of these new resources.

Biclustering Models for Structured Microarray Data

Microarrays have become a standard tool for investigating gene function and more complex microarray experiments are increasingly being conducted. For example, an experiment may involve samples from several groups or may investigate changes in gene expression over time for several subjects, leading to large three-way data sets. In response to this increase in data complexity, we propose some extensions to the plaid model, a biclustering method developed for the analysis of gene expression data. This model-based method lends itself to the incorporation of any additional structure such as external grouping or repeated measures. We describe how the extended models may be fitted and illustrate their use on real data.

Reusing and Modifying Rulebases by Predicate Substitution

We propose a method for reusing and modifying a deductive database. The need for such techniques occurs when new rulebased applications differ only slightly from existing ones or when an application is to be incrementally updated. Such techniques are particularly important when reprogramming is expensive or unreliable. In order to facilitate reuse, we extend deductive database systems by the concept ofpredicate substitution. In this way, during query evaluation, not only variables, but also predicates can be substituted. We provide a proof theory and a model theory for this language, including a fixpoint semantics. In addition, we show that substitution increases the expressive power of Datalog: not only does its data complexity increase from PTIME to EXPTIME, but substitution also allows large sets of Datalog rules to be succinctly expressed. In fact, finite rulebases with substitution can express infinite rulebases without substitution