Data Analysis Platform Research Articles

Spatiotemporal polynomial graph neural network for anomaly detection of complex systems

Internet of Things systems are developed rapidly, there is a need for anomaly detection to identify malicious data to guarantee that the systems perform under healthy condition. However, the dynamic system state is not only influenced by sequential patterns in the time dimension but also affected by other sensor channels in the spatial dimension. Although many successful models have been established in past research to handle multivariate time series data, most of these models exhibit shortcomings in capturing spatiotemporal dependencies. The Internet of Things offers a promising platform for advanced data analysis, but its unique characteristics present challenges for anomaly detection. This paper introduces a novel spatiotemporal polynomial graph neural network (STPGNN) is proposed for anomaly detection of complex systems within the Internet of Things (IoT) framework. Specifically, we propose an adaptive spatiotemporal feature modeling module that includes two developed adaptive modules: the Adaptive Temporal Module and the Spatial Module. These modules are designed to capture dynamic spatial dependencies across multiple time steps and temporal dependencies over time, respectively. The results are then output through a decoder. Importantly, the multi-head attention mechanism employed in both sub-modules can effectively explore potential spatiotemporal dependency patterns within different subspaces. The proposed model, utilizing the innovative approach of STPGNN, demonstrates superior performance compared to existing techniques, as evidenced by experimental results on the real-world test run dataset. This marks a significant advancement in the field of anomaly detection, particularly for complex systems.

Enhancing Performance Metrics: A Google Looker Studio Approach to Key Performance Indicator (KPI) Management System for Homecorp Offshore Drafting Team

The Homecorp Offshore Drafting Key Performance Indicator (KPI) Management System is a comprehensive process that aims to improve the management and monitoring of key performance indicators within the drafting department. This system utilizes Google Looker Studio, which provides a user-friendly real-time data analysis and visualization platform. It enables managers to make well-informed decisions and enhance departmental performance. The process begins with collecting daily task data, logged manually or through automated Google Apps Scripts into a Google Sheets database. This data includes task details such as type, status, and completion time. A KPI-pointing system, developed through focus group discussions, also assigns point values to tasks based on their complexity and time requirements. Integrating Google Looker Studio with Google Sheets allows real-time synchronization and web-based monitoring of key performance indicators (KPIs). The dashboard offers detailed insights such as task counts, average KPIs, and monthly performance graphs, providing a comprehensive performance evaluation. The implementation of this system has been well-received by the team, with survey results indicating improvements in tracking performance metrics, ease of use, and enhanced collaboration. While these outcomes are positive, there is still a recognized need for further refinement, particularly in incorporating qualitative data and improving IT support. The KPI Management System has significantly enhanced performance monitoring and management for the drafting team. To keep the system up-to-date and effective, ongoing feedback and iterative improvements must be maintained. Potential future enhancements could include extending the system's use company-wide to foster a culture of continuous improvement and accountability.

Advanced Data Extraction and Analytics Software on AWS: Revolutionizing Insights with Personalized

Abstract: In the current evolving landscape of marketing, a robust analytics platform is required that integrates data from various marketing channels for insightful decision-making. This paper introduces an Integrated Marketing Data Analytics Platform, a comprehensive solution designed to enhance marketing strategies through advanced analytics. The platform amalgamates data from different marketing sources, enabling an integrated view of campaigns, customer behavior, and market trends. Key features include cross-channel analytics, predictive modeling, and real-time reporting. This research aims to empower marketing professionals with a versatile tool that maximizes the impact and outcome of marketing efforts by providing actionable insights and promoting data-driven decision-making in marketing strategy. (Abstract) Keywords: Marketing analytics, Integrated marketing data , Advanced analytics Cross-ch

Developed Status and Forecast of Intelligent Ship Application

The concept of smart ships is not limited to ship autonomous driving, it covers a broader scope, including the full life cycle of ship design, construction, operation and maintenance. By integrating advanced sensors, navigation systems and data analysis platforms, smart ships are able to monitor ship status and the Marine environment in real time, optimize route planning and improve fuel efficiency, thereby reducing energy consumption and greenhouse gas emissions. In addition, smart ships can revolutionize the shipping industry by improving maritime safety and reducing human error through remote control and autonomous decision-making. In recent years, smart ship technology has gradually moved from the theoretical research stage to practical application, and has attracted wide attention from the global shipping industry. A number of international cooperation and research projects have been launched to explore the practical application potential and promotion paths of smart ships. However, the development of smart ships also faces many challenges, including technical difficulties, legal and regulatory restrictions, and public acceptance issues. This paper will deeply discuss the current development status of smart ships, analyze the main challenges it faces, and forecast the future development trend, in order to provide reference and inspiration for the innovative development of the shipping industry.

AntDAS-GCMS: A New Comprehensive Data Analysis Platform for GC-MS-Based Untargeted Metabolomics with the Advantage of Addressing the Time Shift Problem.

Over the years, a number of state-of-the-art data analysis tools have been developed to provide a comprehensive analysis of data collected from gas chromatography-mass spectrometry (GC-MS). Unfortunately, the time shift problem remains unsolved in these tools. Here, we developed a novel comprehensive data analysis strategy for GC-MS-based untargeted metabolomics (AntDAS-GCMS) to perform total ion chromatogram peak detection, peak resolution, time shift correction, component registration, statistical analysis, and compound identification. Time shift correction was specifically optimized in this work. The information on mass spectra and elution profiles of compounds was used to search for inherent landmarks within analyzed samples to resolve the time shift problem across samples efficiently and accurately. The performance of our AntDAS-GCMS was comprehensively investigated by using four complex GC-MS data sets with various types of time shift problems. Meanwhile, AntDAS-GCMS was compared with advanced GC-MS data analysis tools and classic time shift correction methods. Results indicated that AntDAS-GCMS could achieve the best performance compared to the other methods.

Local outlier factor for anomaly detection in HPCC systems

Local Outlier Factor (LOF) is an unsupervised anomaly detection algorithm that finds anomalies by assessing the local density of a data point relative to its neighborhood. Anomaly detection is the process of finding anomalies in datasets. Anomalies in real-time datasets may indicate critical events like bank frauds, data compromise, network threats, etc. This paper deals with the implementation of the LOF algorithm in the HPCC Systems platform, which is an open-source distributed computing platform for big data analytics. Improved LOF is also proposed which efficiently detects anomalies in datasets rich in duplicates. The impact of varying hyperparameters on the performance of LOF is examined in HPCC Systems. This paper examines the performance of LOF with other algorithms like COF, LoOP, and kNN over several datasets in the HPCC Systems. Additionally, the efficacy of LOF is evaluated across big-data frameworks such as Spark, Hadoop, and HPCC Systems, by comparing their runtime performances.

The tidyomics ecosystem: Enhancing omic data analyses.

The growth of omic data presents evolving challenges in data manipulation, analysis, and integration. Addressing these challenges, Bioconductor1 provides an extensive community-driven biological data analysis platform. Meanwhile, tidy R programming2 offers a revolutionary standard for data organisation and manipulation. Here, we present the tidyomics software ecosystem, bridging Bioconductor to the tidy R paradigm. This ecosystem aims to streamline omic analysis, ease learning, and encourage cross-disciplinary collaborations. We demonstrate the effectiveness of tidyomics by analysing 7.5 million peripheral blood mononuclear cells from the Human Cell Atlas3, spanning six data frameworks and ten analysis tools.

Accelerating Single-Cell Sequencing Data Analysis with SciDAP: A User-Friendly Approach.

Single-cell (sc) RNA, ATAC and Multiome sequencing became powerful tools for uncovering biological and disease mechanisms. Unfortunately, manual analysis of sc data presents multiple challenges due to large data volumes and complexity of configuration parameters. This complexity, as well as not being able to reproduce a computational environment, affects the reproducibility of analysis results. The Scientific Data Analysis Platform (https://SciDAP.com) allows biologists without computational expertise to analyze sequencing-based data using portable and reproducible pipelines written in Common Workflow Language (CWL). Our suite of computational pipelines addresses the most common needs in scRNA-Seq, scATAC-Seq and scMultiome data analysis. When executed on SciDAP, it offers a user-friendly alternative to manual data processing, eliminating the need for coding expertise. In this protocol, we describe the use of SciDAP to analyze scMultiome data. Similar approaches can be used for analysis of scRNA-Seq, scATAC-Seq and scVDJ-Seq datasets.

Big data processing and analysis platform based on deep neural network model

Users are increasingly turning to big data processing systems to extract valuable information from massive datasets as the field of big data grows. Data analytics platforms are used by e-commerce enterprises to improve product suggestions and model business processes. In order to meet the needs of large-scale data center operation and maintenance management, Internet companies often use Flink to process log data. This paper takes the big data processing and analysis platforms built by Internet financial companies and large banks as examples, and implants a stock prediction model based on Deep Neural Network (DNN). In this context, this paper completes the following work: 1) The research status of big data processing and analysis platforms at home and abroad is introduced. 2) Drawing on the modular design idea, the commercial bank big data platform is designed and the functions of each sub-module are introduced. Then the basic principle and structure of Convolutional Neural Networks (CNN) are expounded. 3) The optimal parameters of Convolutional Neural Networks are selected through experiments, and then the trained model is used for experiments. It can be seen that the stock prediction model proposed in this article has a higher prediction accuracy compared to existing models, which also verifies the validity of the proposed model. Input the data and compare the obtained results with the actual results, and finally show that the model in this paper has a good performance on stock prediction.

OHDSI-compliance: a set of document templates facilitating the implementation and operation of a software stack for real-world evidence generation.

The open-source software offered by the Observational Health Data Science and Informatics (OHDSI) collective, including the OMOP-CDM, serves as a major backbone for many real-world evidence networks and distributed health data analytics platforms. While container technology has significantly simplified deployments from a technical perspective, regulatory compliance can remain a major hurdle for the setup and operation of such platforms. In this paper, we present OHDSI-Compliance, a comprehensive set of document templates designed to streamline the data protection and information security-related documentation and coordination efforts required to establish OHDSI installations. To decide on a set of relevant document templates, we first analyzed the legal requirements and associated guidelines with a focus on the General Data Protection Regulation (GDPR). Moreover, we analyzed the software architecture of a typical OHDSI stack and related its components to the different general types of concepts and documentation identified. Then, we created those documents for a prototypical OHDSI installation, based on the so-called Broadsea package, following relevant guidelines from Germany. Finally, we generalized the documents by introducing placeholders and options at places where individual institution-specific content will be needed. We present four documents: (1) a record of processing activities, (2) an information security concept, (3) an authorization concept, as well as (4) an operational concept covering the technical details of maintaining the stack. The documents are publicly available under a permissive license. To the best of our knowledge, there are no other publicly available sets of documents designed to simplify the compliance process for OHDSI deployments. While our documents provide a comprehensive starting point, local specifics need to be added, and, due to the heterogeneity of legal requirements in different countries, further adoptions might be necessary.

Theorizing data analysis platforms – digital refractions and reconfigurations of pasts, presents and futures

ABSTRACT Digital platforms are a pervasive force in the transformation of business, organization and social order, but also an evolving phenomenon in need of further conceptualization and detailed analysis. This article offers a theorization and analytical framework for the study of a sub-type of digital platforms that we may think of as data analysis platforms, and which play an increasingly important role in organizations, business, and society more broadly. Starting from a theoretical focus on how data, algorithms and technological systems work as ‘digital prisms’ that refract and reconfigure social phenomena, the article articulates how data integration and data-driven prediction can be seen as forms of knowledge production that allow for new dynamics of seeing, knowing and governing. On this backdrop, the article highlights five analytical dimensions of data analysis platforms (domains, datasets, models, humans and temporality) and offers a conceptualization of three different types and uses of data analysis platforms (data merging platforms, data prognosis platforms and data projection platforms). Taken together, this analytical framework may guide conceptual and empirical investigations and pave the way for more nuanced understandings of the shape, workings and societal ramifications of data analysis platforms. The article contributes to emergent research seeking to articulate the shapes and workings of digital platforms. The conceptualization of digital platforms may help us understand as the digitalization and datafication of knowledge production, work and organization, the reliance on platforms, as well as broader questions about how data, algorithms and digital systems are used in attempts to grasp and shape futures.

Data analytics for predicting disease outbreaks: A review of models and tools

The burgeoning field of data analytics has emerged as a pivotal force in the realm of public health, particularly in the context of predicting and mitigating disease outbreaks. This comprehensive review delves into the diverse landscape of models and tools employed in data analytics for disease outbreak prediction. With a focus on synthesizing existing knowledge, the paper aims to provide a nuanced understanding of the strengths, limitations, and future directions within this dynamic field. The review begins with an exploration of various models utilized for disease outbreak prediction, ranging from statistical approaches to machine learning models and epidemiological frameworks. Each model category is scrutinized for its efficacy in capturing the complexities inherent in infectious disease dynamics. Simultaneously, the paper investigates the array of tools and technologies leveraged in disease outbreak prediction, encompassing Geographic Information Systems (GIS), data visualization tools, and big data analytics platforms. A critical aspect of the review lies in the examination of diverse data sources contributing to predictive analytics. Epidemiological data, environmental factors, and the burgeoning influence of social media and web data are dissected for their roles in enhancing the accuracy and timeliness of outbreak predictions. Amidst the promises of data analytics, the paper navigates the challenges inherent in predicting disease outbreaks. Issues of data quality and availability, model complexity, interpretability, and ethical considerations are dissected, providing a holistic view of the hurdles that practitioners encounter. Drawing upon case studies and real-world applications, the review showcases instances where data analytics has proven successful in predicting disease outbreaks, shedding light on both triumphs and setbacks. The implications for public health, lessons learned from challenges, and the evolving role of data analytics in shaping global health preparedness are thoroughly discussed. As the paper unfolds, it illuminates future trends and innovations in the field, foreseeing the integration of advanced technologies, global collaboration for information sharing, and the adaptation of predictive analytics for emerging diseases. The review culminates in a comprehensive conclusion, summarizing key findings and emphasizing the potential transformative impact of data analytics on the landscape of disease outbreak prediction.

INSaFLU-TELEVIR: an open web-based bioinformatics suite for viral metagenomic detection and routine genomic surveillance

BackgroundImplementation of clinical metagenomics and pathogen genomic surveillance can be particularly challenging due to the lack of bioinformatics tools and/or expertise. In order to face this challenge, we have previously developed INSaFLU, a free web-based bioinformatics platform for virus next-generation sequencing data analysis. Here, we considerably expanded its genomic surveillance component and developed a new module (TELEVIR) for metagenomic virus identification.ResultsThe routine genomic surveillance component was strengthened with new workflows and functionalities, including (i) a reference-based genome assembly pipeline for Oxford Nanopore technologies (ONT) data; (ii) automated SARS-CoV-2 lineage classification; (iii) Nextclade analysis; (iv) Nextstrain phylogeographic and temporal analysis (SARS-CoV-2, human and avian influenza, monkeypox, respiratory syncytial virus (RSV A/B), as well as a “generic” build for other viruses); and (v) algn2pheno for screening mutations of interest. Both INSaFLU pipelines for reference-based consensus generation (Illumina and ONT) were benchmarked against commonly used command line bioinformatics workflows for SARS-CoV-2, and an INSaFLU snakemake version was released. In parallel, a new module (TELEVIR) for virus detection was developed, after extensive benchmarking of state-of-the-art metagenomics software and following up-to-date recommendations and practices in the field. TELEVIR allows running complex workflows, covering several combinations of steps (e.g., with/without viral enrichment or host depletion), classification software (e.g., Kaiju, Kraken2, Centrifuge, FastViromeExplorer), and databases (RefSeq viral genome, Virosaurus, etc.), while culminating in user- and diagnosis-oriented reports. Finally, to potentiate real-time virus detection during ONT runs, we developed findONTime, a tool aimed at reducing costs and the time between sample reception and diagnosis.ConclusionsThe accessibility, versatility, and functionality of INSaFLU-TELEVIR are expected to supply public and animal health laboratories and researchers with a user-oriented and pan-viral bioinformatics framework that promotes a strengthened and timely viral metagenomic detection and routine genomics surveillance. INSaFLU-TELEVIR is compatible with Illumina, Ion Torrent, and ONT data and is freely available at https://insaflu.insa.pt/ (online tool) and https://github.com/INSaFLU (code).

From Lima to Canton and Beyond: Mobile and Digital Research Infrastructure for Closing the Gap Between Resource-rich and Resource-poor Organisations

ABSTRACT Access to research infrastructure for heritage science is the best way of narrowing the collections knowledge gap between resource-rich and resource-poor cultural institutions. Here we demonstrate a novel workflow that streamlines the use of a mobile laboratory for data collection (MOLAB) followed by a digital platform for data processing, analysis and interpretation (DIGILAB). The data collection and processing are taken care of by heritage science specialists, while the data analysis and interpretation can be performed collaboratively with conservators and curators. MOLAB with specialist personnel and mobile scientific instruments can sometimes be aided by in-house fixed laboratory (FIXLAB) equipment and personnel from resource-rich institutions to assist the resource-poor institutions if their collection can be hosted by the resource-rich institutions for analysis. The aim is to widen participation in the adoption of heritage science and material analysis in the understanding and conservation of collections, while ensuring high-quality, efficient and reliable data acquisition, processing, analysis and interpretation. The approach taken in this project combines access to various research infrastructure platforms to achieve extra benefit through adding further expert-assisted data analysis and interpretation capabilities, enabled by a new digital platform for data processing and visualisation.

An ultra energy-efficient hardware platform for neuromorphic computing enabled by 2D-TMD tunnel-FETs

Brain-like energy-efficient computing has remained elusive for neuromorphic (NM) circuits and hardware platform implementations despite decades of research. In this work we reveal the opportunity to significantly improve the energy efficiency of digital neuromorphic hardware by introducing NM circuits employing two-dimensional (2D) transition metal dichalcogenide (TMD) layered channel material-based tunnel-field-effect transistors (TFETs). Our novel leaky-integrate-fire (LIF) based digital NM circuit along with its Hebbian learning circuitry operates at a wide range of supply voltages, frequencies, and activity factors, enabling two orders of magnitude higher energy-efficient computing that is difficult to achieve with conventional material and/or device platforms, specifically the silicon-based 7 nm low-standby-power FinFET technology. Our innovative 2D-TFET based NM circuit paves the way toward brain-like energy-efficient computing that can unleash major transformations in future AI and data analytics platforms.

Multi-Dimensional Data Analysis Platform (MuDAP): A Cognitive Science Data Toolbox

Researchers in cognitive science have long been interested in modeling human perception using statistical methods. This requires maneuvers because these multiple dimensional data are always intertwined with complex inner structures. The previous studies in cognitive sciences commonly applied principal component analysis (PCA) to truncate data dimensions when dealing with data with multiple dimensions. This is not necessarily because of its merit in terms of mathematical algorithm, but partly because it is easy to conduct with commonly accessible statistical software. On the other hand, dimension reduction might not be the best analysis when modeling data with no more than 20 dimensions. Using state-of-the-art techniques, researchers in various research disciplines (e.g., computer vision) classified data with more than hundreds of dimensions with neural networks and revealed the inner structure of the data. Therefore, it might be more sophisticated to process human perception data directly with neural networks. In this paper, we introduce the multi-dimensional data analysis platform (MuDAP), a powerful toolbox for data analysis in cognitive science. It utilizes artificial intelligence as well as network analysis, an analysis method that takes advantage of data symmetry. With the graphic user interface, a researcher, with or without previous experience, could analyze multiple dimensional data with great ease.

텍스트마이닝 기반 데이터분석을 통한 전문대학 교양교과에 대한 사회적 담론과 현황이해

Objectives The purposes of this study, we seek to gain a macroscopic understanding of the social discourse and current status of liberal arts subjects in colleges in order to improve the quality and strengthen the liberal arts education in colleges.
 Methods To this end, ‘Big data’ was collected using the keywords of ‘junior college+liberal arts’ subjects appearing on the portal site, and data analysis was conducted using the liberal arts subject schedule of junior colleges in the Chungcheong and Gangwon regions as ‘retained data’. Keyword analysis, TF-IDF weight analysis, and network centrality analysis were performed in the big data analysis platform(TEXTOM), and CONCOR analysis was performed using the UCINET6.0 program.
 Results First, as a result of keyword analysis, keywords such as credits, graduation, and completion appeared in portal-collected big data, and in the case of retained data, keywords such as life, English, ability, and understanding were mainly found. Second, keywords with high TF-IDF values a ppeared in the order of ‘credits’, ‘bachelor’s degree’, ‘degree’, and ‘graduation’ in the case of portal-collected big data, and there appeared to be no difference in the case of retained data. Third, as a result of keyword centrality analysis, it was found that in the case of portal big data, ‘credit’ was out of the ranking in closeness centrality, and in the case of retained data, ‘English’ was found to be out of the ranking. Fourth, the clusters formed through CONCOR analysis were found to form two clusters for portal big data and four clusters for retained data.
 Conclusions Big data on liberal arts subjects at colleges showed that social discourse is being formed at a very formal level, such as basic materials related to bachelor's degrees, and the data shows that the nature of liberal arts subjects based on vocational fundamentals is still strong in the composition of subjects.

Analysis and Design of Wind Turbine Monitoring System Based on Edge Computing

INTRODUCTION: A wind turbine data analysis method based on the combination of Hadoop and edge computing is proposed. OBJECTIVES: Solve the wind turbine health status monitoring system large data, time extension, energy consumption and other problems. METHODS: By analysing the technical requirements and business processes of the system, the overall framework of the system was designed and a deep reinforcement learning algorithm based on big data was proposed. RESULTS: It solves the problem of insufficient computing resources as well as energy consumption and latency problems occurring in the data analysis layer, solves the problems in WTG task offloading, and improves the computational offloading efficiency of the edge nodes to complete the collection, storage, and analysis of WTG data. CONCLUSION: The data analysis and experimental simulation platform is built through Python, and the results show that the application of Hadoop and the edge computing offloading strategy based on the DDPG algorithm to the system improves the system's quality of service and computational performance, and the method is applicable to the distributed storage and analysis of the device in the massive monitoring data.

MetaboAnalyst 6.0: towards a unified platform for metabolomics data processing, analysis and interpretation.

We introduce MetaboAnalyst version 6.0 as a unified platform for processing, analyzing, and interpreting data from targeted as well as untargeted metabolomics studies using liquid chromatography - mass spectrometry (LC-MS). The two main objectives in developing version 6.0 are to support tandem MS (MS2) data processing and annotation, as well as to support the analysis of data from exposomics studies and related experiments. Key features of MetaboAnalyst 6.0 include: (i) a significantly enhanced Spectra Processing module with support for MS2 data and the asari algorithm; (ii) a MS2 Peak Annotation module based on comprehensive MS2 reference databases with fragment-level annotation; (iii) a new Statistical Analysis module dedicated for handling complex study design withmultiple factors orphenotypic descriptors; (iv) a Causal Analysis module for estimating metabolite - phenotype causal relations based on two-sample Mendelian randomization,and (v) a Dose-Response Analysis module for benchmark dose calculations. In addition, we have also improved MetaboAnalyst's visualization functions, updated its compound database and metabolite sets, and significantly expanded its pathway analysis support to around 130 species. MetaboAnalyst 6.0 is freely available at https://www.metaboanalyst.ca.

ASIVA – Platform for observational and computational analysis of stellar variables

The astronomical data analysis consists of two crucial process; data reduction of the captured images and data analysis of the derived magnitudes. We present the platform ASIVA, a data analysis platform which comes along with a data reduction pipeline. The data reduction pipeline gives flexibility to analyse the FITS images and also perform image alignment for detecting the correct image coordinates for required objects. It can be custom scheduled with cron jobs so that it picks the latest data and appends the results accordingly. The data analysis platform allows user to effectively analyse the ensemble data and perform accurate data processing and grouping with ease. It is integrated with a custom algorithm to detect the variable stars from an ensemble with its relative standard deviations. The statistical, spectral and non-linear dynamics features can be extracted from time series data which can be eventually used for in-depth analysis. To validate the capability, we have analysed 15 nights of Orion Nebula Cluster field in I filter which had 1585 images. ASIVA reduces manual effort to a great extent thus saves analysis time and excludes human errors.