Analysis Of Heterogeneous Data Research Articles

Research and Application of Spacecraft Product Inspection System Based on Smart Glasses

cenarios, studies the construction of the digital test system based on intelligent glasses, based on the standard operating procedure operation guide, inspection data online and conformity intelligent decision, heterogeneous data packet generation and analysis of key technology, realized the application of spacecraft product inspection process, improves the working efficiency and quality.

An analytic framework for enhancing the performance of big heterogeneous data analysis

The use of social media networks is becoming a current phenomenon in the world today where people are sharing posts and tweets, connect with different groups, and share their opinions about things. This data is extremely heterogeneous and so it is hard to analyze and derive information from this data that is considered an indispensable source for decision-makers. New techniques are therefore needed to handle these huge amounts of data to find the hidden information thus improve the results of the analysis. We are developing a framework for the analysis of heterogeneous data using machine learning (ML) techniques. In contrast to most of the literature frameworks that focus on a specific type of heterogeneous data for evaluating the proposed framework, we have analyzed 15k tweets data from six American airlines. These tweets are collected from the open stream of Twitter, also predict, classify each tweet as a negative or positive review, and test the ability of deep learning (DL) algorithms by comparing it with traditional ML algorithms. The findings confirmed the validity of the proposed framework and helped to achieve the study objective by providing excellent analysis performance and provide insights into additional aspects of information extraction from heterogeneous data.

A Multi-Order Distributed HOSVD with Its Incremental Computing for Big Services in Cyber-Physical-Social Systems

Big service is an extremely important application of service computing to provide predictive and needed services to humans. To operationalize big services, the heterogeneous data collected from Cyber-Physical-Social Systems (CPSS) must be processed efficiently. However, because of the rapid rise in the volume of data, faster and more efficient computational techniques are required. Therefore, in this paper, we propose a multi-order distributed high-order singular value decomposition method (MDHOSVD) with its incremental computational algorithm. To realize the MDHOSVD, a tensor blocks unfolding integration regulation is proposed. This method allows for the efficient analysis of large-scale heterogeneous data in blocks in an incremental fashion. Using simulation and experimental results from real-life, the high-efficiency of the proposed data processing and computational method, is demonstrated. Further, a case study about cyber-physical-social system data processing is illustrated. The proposed MDHOSVD method speeds up data processing, scales with data volume, improves the adaptability and extensibility over data diversity and converts low-level data into actionable knowledge.

Abstract 14868: Promises of Transcatheter Tricuspid Valvular Interventions in Patients With Clinically Significant Tricuspid Regurgitation: A Meta-analysis

Introduction: Clinically significant tricuspid regurgitation is common especially in patients with structural heart disease and is independently associated with poor outcomes. Several transcatheter interventions for tricuspid valve repair are under investigation. We aimed to illustrate and summarize the efficacy and safety of these interventions. Methods: We conducted a literature search for relevant clinical registries and trials published until April 2020. We searched PubMed, Scopus, Web of Science, and Cochrane CENTRAL. We performed the analysis of continuous outcomes using the mean difference (MD) while used the risk ratio (RR) for analysis of dichotomous outcomes. We included the following outcomes: Tricuspid annular plane systolic excursion (TAPSE), Vena Contracta (VC), Tricuspid Annulus diameter (TAD), Tricuspid regurgitation volume (TRV), Left ventricular ejection fraction (LVEF), and mortality rates. The random-effects model was used for the analysis of heterogeneous data, while homogeneous data were analyzed under the fixed-effects model. Results: We obtained fifteen clinical registries/trials and included a total of 1329 patients. We found that tricuspid valve interventional devices improved LVEF (MD = 1.46%, 95% CI [0.45, 2.47], p=0.005), led to smaller VC (MD = -4.69, 95% CI [-5.50, -3.88], p=0.005), lower TRV (MD = -11.68, 95% CI [-21.25, -2.11], p=0.02), and smaller TAD (MD = -3.78mm, 95% CI [-5.09, -2.47], p<0.001). No significant difference was noted regarding TAPSE (MD=0.07mm, 95% CI [-0.29, 0.44], p=0.7). Results of the six minutes walking test showed that patients could walk 40 more meters after intervention (MD = 40.64, 95% CI [23.89, 57.40], p<0.001). Pooled mortality rate was 8.2%, stroke 1%, and bleeding 5% during the mean 12 months follow up. Conclusions: Transcatheter tricuspid valve repair devices prove to be effective interventions in decreasing regurgitation and improving functional capacity in patients with clinically significant TR. Mortality (8% during mean 12 months follow up) seems to be related to underlying cardiovascular disease.

IODA: An integrated tool for analysis of cancer pathway consistency from heterogeneous multi-omics data

The latest advances in the next generation sequencing technology have greatly facilitated the extensive research of genomics and transcriptomics, thereby promoting the decoding of carcinogenesis with unprecedented resolution. Considering the contribution of analyzing high-throughput multi-omics data to the exploration of cancer molecular mechanisms, an integrated tool for heterogeneous multi-omics data analysis (iODA) is proposed for the systems-level interpretation of multi-omics data, i.e., transcriptomic profiles (mRNA or miRNA expression data) and protein-DNA interactions (ChIP-Seq data). Considering the data heterogeneity, iODA can compare six statistical algorithms in differential analysis for the selected sample data and assist users in choosing the globally optimal one for dysfunctional mRNA or miRNA identification. Since molecular signatures are more consistent at the pathway level than at the gene level, the tool is able to enrich the identified dysfunctional molecules onto the KEGG pathways and extracted the consistent items as key components for further pathogenesis investigation. Compared with other tools, iODA is multi-functional for the systematic analysis of different level of omics data, and its analytical power was demonstrated through case studies of single and cross-level prostate cancer omics data. iODA is open source under GNU GPL and can be downloaded from http://www.sysbio.org.cn/iODA.

Intelligent analytical system as a tool to ensure the reproducibility of biomedical calculations

The experience of the use of applied containerized biomedical software tools in cloud environment is summarized. The reproducibility of scientific computing in relation with modern technologies of scientific calculations is discussed. The main approaches to biomedical data preprocessing and integration in the framework of the intelligent analytical system are described. At the conditions of pandemic, the success of health care system depends significantly on the regular implementation of effective research tools and population monitoring. The earlier the risks of disease can be identified, the more effective process of preventive measures or treatments can be. This publication is about the creation of a prototype for such a tool within the project «Development of methods, algorithms and intelligent analytical system for processing and analysis of heterogeneous clinical and biomedical data to improve the diagnosis of complex diseases» (M/99-2019, M/37-2020 with support of the Ministry of Education and Science of Ukraine), implementted by the V.M. Glushkov Institute of Cybernetics, National Academy of Sciences of Ukraine, together with the United Institute of Informatics Problems, National Academy of Sciences of Belarus (F19UKRG-005 with support of the Belarussian Republican Foundation for Fundamental Research). The insurers, entering the market, can insure mostly low risks by facilitating more frequent changes of insurers by consumers (policyholders) and mixing the overall health insurance market. Socio-demographic variables can be risk adjusters. Since age and gender have a relatively small explanatory power, other socio-demographic variables were studied – marital status, retirement status, disability status, educational level, income level. Because insurers have an interest in beneficial diagnoses for their policyholders, they are also interested in the ability to interpret relevant information – upcoding: insurers can encourage their policyholders to consult with doctors more often to select as many diagnoses as possible. Many countries and health care systems use diagnostic information to determine the reimbursement to a service provider, revealing the necessary data. For processing and analysis of these data, software implementations of construction for classifiers, allocation of informative features, processing of heterogeneous medical and biological variables for carrying out scientific research in the field of clinical medicine are developed. The experience of the use of applied containerized biomedical software tools in cloud environment is summarized. The reproducibility of scientific computing in relation with modern technologies of scientific calculations is discussed. Particularly, attention is paid to containerization of biomedical applications (Docker, Singularity containerization technology), this permits to get reproducibility of the conditions in which the calculations took place (invariability of software including software and libraries), technologies of software pipelining of calculations, that allows to organize flow calculations, and technologies for parameterization of software environment, that allows to reproduce, if necessary, an identical computing environment. The main approaches to biomedical data preprocessing and integration in the framework of the intelligent analytical system are described. The experience of using the developed linear classifier, gained during its testing on artificial and real data, allows us to conclude about several advantages provided by the containerized form of the created application: it permits to provide access to real data located in cloud environment; it is possible to perform calculations to solve research problems on cloud resources both with the help of developed tools and with the help of cloud services; such a form of research organization makes numerical experiments reproducible, i.e. any other researcher can compare the results of their developments on specific data that have already been studied by others, in order to verify the conclusions and technical feasibility of new results; there exists a universal opportunity to use the developed tools on technical devices of various classes from a personal computer to powerful cluster.

A study on a general cyber machine tools monitoring system in smart factories

The massive data produced in real manufacturing processes provides a micro observation mechanism for managers and producers, which helps to analyze and grasp the manufacturing details, to form a macro decision-making mechanism, and finally to improve the manufacturing quality and production efficiency. This paper deeply investigates the real-time acquisition method of machine tool processing data and presents a general method of data analysis in different actual application scenarios to solve the general problem of insufficient and inappropriate utilization of machine tools. A framework of general machine tool monitoring system with four layers is proposed and several key stumbling blocks are researched, such as heterogeneous machine data acquisition, data analysis, and related application. These achievements are applied in a general cyber machine tools monitoring system based on MTConnect, which has greatly improved the utilization of machine tools and become a valuable unit in the framework of intelligent manufacturing and smart factory.

Cardiac radiomics: an interactive approach for 4D data exploration

Abstract Cardiac diseases manifest in a multitude of interconnected changes in morphology and dynamics. Radiomics approaches are a promising technique to analyze such changes directly from image data. We propose novel features to specifically describe moving cardiac structures, and an interactive 4D visualization method to explore such data. Prototypical tests with an open data set containing different diseases show that our approach can be a fast and useful tool for the 4D analysis of heterogeneous cohort data.

Mimicry Embedding Facilitates Advanced Neural Network Training for Image-Based Pathogen Detection.

The use of deep neural networks (DNNs) for analysis of complex biomedical images shows great promise but is hampered by a lack of large verified data sets for rapid network evolution. Here, we present a novel strategy, termed "mimicry embedding," for rapid application of neural network architecture-based analysis of pathogen imaging data sets. Embedding of a novel host-pathogen data set, such that it mimics a verified data set, enables efficient deep learning using high expressive capacity architectures and seamless architecture switching. We applied this strategy across various microbiological phenotypes, from superresolved viruses to in vitro and in vivo parasitic infections. We demonstrate that mimicry embedding enables efficient and accurate analysis of two- and three-dimensional microscopy data sets. The results suggest that transfer learning from pretrained network data may be a powerful general strategy for analysis of heterogeneous pathogen fluorescence imaging data sets.IMPORTANCE In biology, the use of deep neural networks (DNNs) for analysis of pathogen infection is hampered by a lack of large verified data sets needed for rapid network evolution. Artificial neural networks detect handwritten digits with high precision thanks to large data sets, such as MNIST, that allow nearly unlimited training. Here, we developed a novel strategy we call mimicry embedding, which allows artificial intelligence (AI)-based analysis of variable pathogen-host data sets. We show that deep learning can be used to detect and classify single pathogens based on small differences.

Data-Intensive Task Scheduling for Heterogeneous Big Data Analytics in IoT System

Efficient big data analysis is critical to support applications or services in Internet of Things (IoT) system, especially for the time-intensive services. Hence, the data center may host heterogeneous big data analysis tasks for multiple IoT systems. It is a challenging problem since the data centers usually need to schedule a large number of periodic or online tasks in a short time. In this paper, we investigate the heterogeneous task scheduling problem to reduce the global task execution time, which is also an efficient method to reduce energy consumption for data centers. We establish the task execution for heterogeneous tasks respectively based on the data locality feature, which also indicate the relationship among the tasks, data blocks and servers. We propose a heterogeneous task scheduling algorithm with data migration. The core idea of the algorithm is to maximize the efficiency by comparing the cost between remote task execution and data migration, which could improve the data locality and reduce task execution time. We conduct extensive simulations and the experimental results show that our algorithm has better performance than the traditional methods, and data migration actually works to reduce th overall task execution time. The algorithm also shows acceptable fairness for the heterogeneous tasks.

Global Research Alliance N2 O chamber methodology guidelines: Statistical considerations, emission factor calculation, and data reporting.

Static chambers are often used for measuring nitrous oxide (N2 O) fluxes from soils, but statistical analysis of chamber data is challenged by the inherently heterogeneous nature of N2 O fluxes. Because N2 O chamber measurements are commonly used to assess N2 O mitigation strategies or to determine country-specific emission factors (EFs) for calculating national greenhouse gas inventories, it is important that statistical analysis of the data is sound and that EFs are robustly estimated. This paper is one of a series of articles that provide guidance on different aspects of N2 O chamber methodologies. Here, we discuss the challenges associated with statistical analysis of heterogeneous data, by summarizing statistical approaches used in recent publications and providing guidance on assessing normality and options for transforming data that follow a non-normal distribution. We also recommend minimum requirements for reporting of experimental and metadata of N2 O studies to ensure that the robustness of the results can be reliably evaluated. This includes detailed information on the experimental site, methodology and measurement procedures, gas analysis, data and statistical analyses, and approaches to generate EFs, as well as results of ancillary measurements. The reliability, robustness, and comparability of soil N2 O emissions data will be improved through (a) application, and reporting, of more rigorous methodological standards by researchers and (b) greater vigilance by reviewers and scientific editors to ensure that all necessary information is reported in scientific publications.

Approaches for Assessing the Polymetallic Impact on Humans in an Urbanized Area Using Neural Network Modeling Methods

A methodology for assessing the response to the polymetallic effects of sensitive population groups has been developed using new information methods and data collection paths used for evaluation. The presented methodology takes into account the processes of polymeric intake, accumulation in the environment, and biosubstrates of the human body. The approach is presented for simultaneous accounting and analysis of dissimilar and heterogeneous data having different degrees of influence on the content of metals in human biosubstrates and the investigated components of the environment using neural self-organizing network Kohonen, implemented in the author’s model of neuronetwork filtration. A method of determining probability of exceeding the threshold values of metal concentrations in the objects under investigation using the risk theory is proposed. As the thresholds of these events, we have taken the median of continuous series of observations of metal concentrations. The facts of established exceeding the thresholds (events) were recorded in the whole set of observations, allocated in the form of clusters formed by the help of neural network classification. In this case, the spatial distribution of selected observations having specific address mappings forms the corresponding territorial zones, which can be used for operational and planned quality management of environmental components in an urbanized territory.

A Long Short Term Memory with Peephole Connections and Generative Adversarial Network Based Collaborative Methodology to Identify Outliers in ECG Dataset

Big Data analytics has become an upward field, and it plays a pivotal role in Healthcare and research practices. Big data analytics in healthcare cover vast numbers of dynamic heterogeneous data integration and analysis. Medical records of patients include several data including medical conditions, medications and test findings. One of the major challenges of analytics and prediction in healthcare is data preprocessing. In data preprocessing the outlier identification and correction is the important challenge. Outliers are exciting values that deviates from other values of the attribute; they may simply experimental errors or novelty. Outlier identification is the method of identifying data objects with somewhat different behaviors than expectations. Detecting outliers in time series data is different from normal data. Time series data are the data that are in a series of certain time periods. This kind of data are identified and cleared to bring the quality dataset. In this proposed work a hybrid outlier detection algorithm extended LSTM-GAN is helped to recognize the outliers in time series data. The outcome of the proposed extended algorithm attained better enactment in the time series analysis on ECG dataset processing compared with traditional methodologies.

A multi-source heterogeneous data analytic method for future price fluctuation prediction

Most previous works on future market price forecasting only utilize the historical transaction data, while ignoring many other valuable factors. Recently, many research works propose multiple-source data-based predicting approaches in the stock market. Although the futures market and the stock market are very similar, the futures market still has its uniqueness. Most importantly, the subject matter of futures is usually commodity entities with prominent competing products or upstream, downstream industries, which can significantly influence the price. Therefore, it is essential to propose a future specific analysis framework by considering different factors.In this study, we constructed a Multi-source Heterogeneous Data Analysis (MHDA) method for future price prediction by integrating multiple-source information, i.e., trading data, news event data, and investor comments. Firstly, we first constructed a relation map to capture all related news events from upstream and downstream commodities and then built a future-specific sentiment dictionary to accurately quantify the sentiment impact of related news and investor comments during the feature extraction. Finally, we model the quantified multi-source heterogeneous information by an extended Hidden Markov Model to capture the underlying temporal dependency in the data. Evaluations on the data of palm oil futures from 2016.9 to 2017.9 show the effectiveness of our proposed framework.

Data Lake Governance: Towards a Systemic and Natural Ecosystem Analogy

The realm of big data has brought new venues for knowledge acquisition, but also major challenges including data interoperability and effective management. The great volume of miscellaneous data renders the generation of new knowledge a complex data analysis process. Presently, big data technologies provide multiple solutions and tools towards the semantic analysis of heterogeneous data, including their accessibility and reusability. However, in addition to learning from data, we are faced with the issue of data storage and management in a cost-effective and reliable manner. This is the core topic of this paper. A data lake, inspired by the natural lake, is a centralized data repository that stores all kinds of data in any format and structure. This allows any type of data to be ingested into the data lake without any restriction or normalization. This could lead to a critical problem known as data swamp, which can contain invalid or incoherent data that adds no values for further knowledge acquisition. To deal with the potential avalanche of data, some legislation is required to turn such heterogeneous datasets into manageable data. In this article, we address this problem and propose some solutions concerning innovative methods, derived from a multidisciplinary science perspective to manage data lake. The proposed methods imitate the supply chain management and natural lake principles with an emphasis on the importance of the data life cycle, to implement responsible data governance for the data lake.

Spatial Analysis of Asymmetry in the Development of Tourism Infrastructure in the Borderlands: The Case of the Bystrzyckie and Orlickie Mountains

This paper discusses the issue of analyzing the development of cross-border tourism infrastructure in the borderlands of countries with diversified administrative divisions and spatial databases, which hinders the use of national statistical units for comparative research. As an example, the ability to use the square grid and kernel density estimation methods for the analysis and spatial visualization of the level of tourism infrastructure development is studied for the Orlickie and Bystrzyckie Mountains, located in the Polish–Czech border area. To synthetically assess and compare the level of diversity, the methodology used in the Human Development Index was adapted using selected component indicators calculated for a square grid clipped to the boundaries of the area under study. This analysis enabled us to quantify the asymmetry in the development of tourism infrastructure in the borderlands via the calculation of the synthetic infrastructure development index. This index is 1.29 times higher in the Czech than in the Polish border area. However, the spatial concentration analysis of infrastructure shows that the diversity in the study area can be assessed as higher than the results using the average density indicators. This paper also discusses the benefits and problems associated with using the square grid method for the representation and analysis of heterogeneous data on tourism infrastructure in two neighboring national states.

An Integrative Framework of Heterogeneous Genomic Data for Cancer Dynamic Modules Based on Matrix Decomposition.

Cancer progression is dynamic, and tracking dynamic modules is promising for cancer diagnosis and therapy. Accumulated genomic data provide us an opportunity to investigate the underlying mechanisms of cancers. However, as far as we know, no algorithm has been designed for dynamic modules by integrating heterogeneous omics data. To address this issue, we propose an integrative framework for dynamic module detection based on regularized nonnegative matrix factorization method (DrNMF) by integrating the gene expression and protein interaction network. To remove the heterogeneity of genomic data, we divide the samples of expression profiles into groups to construct gene co-expression networks. To characterize the dynamics of modules, the temporal smoothness framework is adopted, in which the gene co-expression network at the previous stage and protein interaction network are incorporated into the objective function of DrNMF via regularization. The experimental results demonstrate that DrNMF is superior to state-of-the-art methods in terms of accuracy. For breast cancer data, the obtained dynamic modules are more enriched by the known pathways, and can be used to predict the stages of cancers and survival time of patients. The proposed model and algorithm provide an effective integrative analysis of heterogeneous genomic data for cancer progression.

A Distributed Stream Processing Middleware Framework for Real-Time Analysis of Heterogeneous Data on Big Data Platform: Case of Environmental Monitoring.

In recent years, the application and wide adoption of Internet of Things (IoT)-based technologies have increased the proliferation of monitoring systems, which has consequently exponentially increased the amounts of heterogeneous data generated. Processing and analysing the massive amount of data produced is cumbersome and gradually moving from classical ‘batch’ processing—extract, transform, load (ETL) technique to real-time processing. For instance, in environmental monitoring and management domain, time-series data and historical dataset are crucial for prediction models. However, the environmental monitoring domain still utilises legacy systems, which complicates the real-time analysis of the essential data, integration with big data platforms and reliance on batch processing. Herein, as a solution, a distributed stream processing middleware framework for real-time analysis of heterogeneous environmental monitoring and management data is presented and tested on a cluster using open source technologies in a big data environment. The system ingests datasets from legacy systems and sensor data from heterogeneous automated weather systems irrespective of the data types to Apache Kafka topics using Kafka Connect APIs for processing by the Kafka streaming processing engine. The stream processing engine executes the predictive numerical models and algorithms represented in event processing (EP) languages for real-time analysis of the data streams. To prove the feasibility of the proposed framework, we implemented the system using a case study scenario of drought prediction and forecasting based on the Effective Drought Index (EDI) model. Firstly, we transform the predictive model into a form that could be executed by the streaming engine for real-time computing. Secondly, the model is applied to the ingested data streams and datasets to predict drought through persistent querying of the infinite streams to detect anomalies. As a conclusion of this study, a performance evaluation of the distributed stream processing middleware infrastructure is calculated to determine the real-time effectiveness of the framework.

Privacy Preserving High-Order Bi-Lanczos in Cloud–Fog Computing for Industrial Applications

Industrial cyber–physical–social systems (CPSSs), a prominent data-driven paradigm, tightly couple and coordinate social space into cyber–physical systems (CPSs) within industrial environments. With the proliferation of cloud–fog computing, cloud–fog computing becomes the most prominent computing paradigm used to implement industrial data analysis. However, the open environment of cloud–fog computing and the limited control of industrial CPSSs users make industrial data analysis without compromising users’ privacy one great research challenge in practical cloud–fog-based industrial applications. High-order Bi-Lanczos (HOBI-Lanczos) approach has shown remarkable success in heterogeneous data analysis in industrial applications. In this article, a novel privacy preserving HOBI-Lanczos approach using tensor train in cloud–fog computing is proposed for industrial data applications. Specifically, a privacy preserving industrial data analysis model using cloud–fog computing and tensor train is firstly proposed. The proposed model enables fogs and clouds to securely carry out industrial data analysis for large-scale tensors given in a tensor train format. In addition, by using this model, a privacy preserving HOBI-Lanczos approach is provided. Last but not least, by using a brain-controlled robot system case study, the proposed approach is theoretically and empirically analyzed. Our proposed approach is proven to be secure. A series of experiments corroborate the superiority of the proposed approach in cloud–fog computing for industrial applications.

3D scene and geological modeling using integrated multi-source spatial data: Methodology, challenges, and suggestions

The visualization and analysis of massive, multi-source, multi-dimensional, heterogeneous, dynamic, and complex spatial data is playing an increasingly important role in 3D scene and geological modeling. There are great differences in various research fields regarding data acquisition techniques, morphological descriptions, and application purposes of spatial objects. Therefore, based on an existing three-dimensional visualization technology and analysis program of spatial data, an integrated processing method for the effective fusion and analysis of multi-source complex spatial data is proposed. This method combines a massive point cloud model with a digital elevation model, in addition to combining the complex stratum with fault modeling technology. The structural grid and geological grid are effectively combined to realize the unified modeling and visualization of natural and artificial objects. This method was then applied to a 3D modeling scene of deeply buried mines and a geological construction model for open pit mines. The comprehensive and effective utilization of multi-source geological data is realized by using this modeling method. Finally, the challenges of spatial data visualization from the aspects of the difficulties in spatial data acquisition, the complexities of the spatial structure, the constraint relations of geological bodies, and the complexity of the 3D modeling process are discussed in this study. Three specific suggestions are proposed. This study enhances the application efficiency of multi-source spatial data modeling technologies already in practice and provides reference for future applications and developments.