Heterogeneous Data Processing Research Articles

Multilayer Spectral Graph Clustering via Convex Layer Aggregation: Theory and Algorithms

Multilayer graphs are commonly used for representing different relations between entities and handling heterogeneous data processing tasks. Nonstandard multilayer graph clustering methods are needed for assigning clusters to a common multilayer node set and for combining information from each layer. This paper presents a multilayer spectral graph clustering (SGC) framework that performs convex layer aggregation. Under a multilayer signal-plus-noise model, we provide a phase transition analysis of clustering reliability. Moreover, we use the phase transition criterion to propose a multilayer iterative model order selection algorithm (MIMOSA) for multilayer SGC, which features automated cluster assignment and layer weight adaptation, and provides statistical clustering reliability guarantees. Numerical simulations on synthetic multilayer graphs verify the phase transition analysis, and experiments on real-world multilayer graphs show that MIMOSA is competitive or better than other clustering methods.

Combining Cloud-based Workflow Management System with SOA and CEP to Create Agility in Collaborative Environment

1 IntroductionDuring its evolution, humanity always looked for instruments and innovative solutions that ensure maximum comfort from the perspective of human universe dimensions: familial, social, cultural, economic, religious etc. Many times during this social development, the individual had to resort to various association forms with others in order to achieve these goals, thus laying the foundation for what we call today collaboration. Current forms of association transcend the original goals and the individual aspirations have been altered and transposed into mercantile organization objectives. Unconditional support of these considerations is reflected by the capitalist system and largely by the industrial system, where the aim of earning money through all means outweighs any attempt of collaboration in the original meaning. Unfortunately the mercantilism is not only an economic doctrine of 15th-18th centuries, but it is still fully manifest in the digital society.Beyond these considerations, thinking of the social and economic status quo we must notice a change in the business paradigm. The traditional models defined through the goal of maximizing the profit adapt to modern management doctrines, oriented towards the individuals. Current society is not only information and knowledge based; it is also a society of technology and artificial intelligence. Transposing the business in the digital environment is no longer a caprice but a need. Additionally, the simple presence in virtual environment is not the premise for the success of modern business, the advantages provided by technology must also be fully exploited.Orientation towards exploiting individual abilities and ensuring business processes flexibility has led to what specialists call business agility. Ensuring a certain level of agility for the business requires more and more relying on information and communications technology. Additionally, the agile business status implies redefining the human resource as primary production factor. The anthropocentric paradigm is evidence to this. In the context of modern agile business collaboration and team work are more than ever necessary. Through the prism of dedicated agility technologies, the essence of collaboration as instrument for founding and unfolding economic activities dissolves into the business itself, becoming visible on the level of business processes and work tasks. The individuals, activities, processes and work tasks are vertices of informational graphs representing the functioning of modern businesses. The edges of these graphs are merely informational flows (later translated into workflows) that ensure communication and cooperation on business level for reaching its goals.For technological and economic reasons (meaning resource economy), the need to share resources, skills, competencies, market shares and experience overlays on top of these modern business architecture. In many occasions this was a premise for development of virtual organizations that take advantage of concrete market opportunities. The modern technological solutions is now provided by cloud computing. Together with a workflow management system (WfMS), cloud computing comes to ensure scalability, reutilization of technological capabilities, reliability, improvement of business processes, minimization of information and communications technology (ICT) investment.Thus, exploitation of cloud services may generate an intensified use of workflows, mainly for collaborative environments. On the other hand, business agility built on service-oriented architecture (SOA) and complex event processing (CEP) solutions may ensure real time collaboration between business processes through processing heterogeneous data extracted from business connected information flows. A solution of workflow management system based on cloud for real time collaboration will provide the capabilities for multi-criterion analyses for business processes, while considering their interdependence with the environment. …

A Model-Driven Methodology for the Design of Autonomic and Cognitive IoT-Based Systems: Application to Healthcare

Due to its abilities to capture real-time data concerning the physical world, the Internet of Things (IoT) phenomenon is fast gaining momentum in different applicative domains. Its benefits are not limited to connecting things, but lean on how the collected data are transformed into insights and interact with domain experts for better decisions. Nonetheless, a set of challenges including the complexity of IoT-based systems and the management of the ensuing big and heterogeneous data and as well as the system scalability need to be addressed for the development of flexible smart IoT-based systems that drive the business decision-making. Consequently, inspired from the human nervous system and cognitive abilities, we have proposed a set of autonomic cognitive design patterns that alleviate the design complexity of smart IoT-based systems, while taking into consideration big data and scalability management. The ultimate goal of these patterns is providing generic and reusable solutions for elaborating flexible smart IoT-based systems able to perceive the collected data and provide decisions. These patterns are articulated within a model-driven methodology that we have proposed to incrementally refine the system functional and nonfunctional requirements. Following the proposed methodology, we have combined and instantiated a set of patterns for developing a flexible cognitive monitoring system to manage patients’ health based on heterogeneous wearable devices. We have highlighted the gained flexibility and demonstrated the ability of our system to integrate and process heterogeneous large-scale data streams. Finally, we have evaluated the system performance in terms of response time and scalability management.

Processing of heterogeneous MS register data within the EUReMS project

ABSTRACT
 ObjectivesEUReMS (European Register for Multiple Sclerosis), a project including more than ten national and regional European MS registers, is aiming to enable analyses across European registers by joining existing, heterogeneous MS data in four different studies. Each participating register delivered productive data comprising information on socio-demography, disease course, medical exams or treatment. In terms of data quality, especially comparability and integrity, a data handling routine has been implemented using an open source ETL (extract transform load) tool (“Talend Open Studio”) to process the large amounts of heterogeneous raw data. That approach will be presented.
 ApproachAs a first step in harmonizing datasets of different registers, a basic EUReMS data structure was defined for each of the four project studies, considering all information required to answer the research questions. Through the data handling process, the data exports are going to be converted into the prior defined study data structure to facilitate comparability and data analyses across the various registers participating in one study. In regard to quality assurance the data handling process has been validated before providing data for analyses.
 ResultsThe data handling process consists of five steps: Reading/Splitting, Cleaning, Mapping and Creating Study Datasets. During the first step, data is read and split into variables that are going to be used within the study datasets. The heterogeneity of the data is again noticeable in the data types of the source files, ranging from csv or Excel to Access Database. During the cleaning step, data is checked for incorrect or missing values and are, as a way of ensuring traceability, saved in specific reject files. In the mapping step, register specific variables are mapped to the defined EUReMS denotations. By that, the heterogeneous data is harmonized, disabling misinterpretation of register specific variables, often in national language or unfamiliar abbreviations. The data is merged into study datasets that are uniform in appearance for each study and are provided to the statistical department for analyses in order to gain insight on disease related questions.
 ConclusionThe implemented process enables the transparent, standardised and reproducible handling of heterogeneous data and is the groundwork for analyses across the various MS registers. Though it is a time-consuming task at the first implementation, we have been able to harmonise the heterogeneous data successfully.

Big media healthcare data processing in cloud: a collaborative resource management perspective

Nowadays, big media healthcare data processing in cloud has become an effective solution for satisfying QoS demands of medical users. It can support various healthcare services such as pre-processing, storing, sharing, and analysis of monitored data as well as acquiring context-awareness. However, to support energy and cost savings, the union of cloud data centers termed as cloud confederation can be an promising approach, which helps a cloud provider to overcome the limitation of physical resources. However, the key challenge in it is to achieve multiple contradictory objectives, e.g., meeting the required level of services defined in service level agreement, maintaining medial users'application QoS, etc. while maximizing profit of a cloud provider. In this paper, for executing heterogeneous big healthcare data processing requests from users, we develop a local and global cloud confederation model, namely FnF, that makes an optimal selection decision for target cloud data center(s) by exploiting Fuzzy logic. The FnF trades off in between profit of cloud provider and user application QoS in selecting federated data center(s). In addition, FnF enhances its decision accuracy by precisely estimating the resource requirements for the big data processing tasks using multiple linear regression. The proposed FnF model is validated through numerical as well as experimental evaluations. Simulation results depict the effectiveness and efficiency of the FnF model compared to state-of-the-art approaches.

A privacy-preserving high-order neuro-fuzzy c-means algorithm with cloud computing

Currently, massive heterogeneous data is generating from the Internet of Things (IoT). Heterogeneous data processing with the neuro-fuzzy technology has become a hot topic for IoT. In this work, we propose a privacy-preserving high-order neuro-fuzzy c-means algorithm for clustering heterogeneous data (PPHOFCM) on cloud computing. PPHOFCM clusters the heterogeneous data set by representing each heterogeneous data object as a tensor and uses the tensor distance to capture the correlations in the high-order tensor space. Furthermore, the cloud computing is employed to improve the clustering efficiency for massive heterogeneous data from IoT. The BGV encryption scheme is used to protect the private data when performing the high-order neuro-fuzzy c-means algorithm on cloud computing. Experiments are conducted on two real IoT datasets to verify the proposed algorithm.

Editorial for the special issue on heterogeneous sensors–based object identification and information fusion

With the fast development of computing, communication, control technologies, the proliferation of heterogeneous sensors with data communication capability can provide a wide variety of data for our daily life, such as road monitoring, health-care systems, structure health checking, military applications etc. However, there are still many interesting open research problems left to be explored as well as many issues to be addressed. In this special issue, we plan to focus on various research and application issues in heterogeneous sensors based object identification and information fusion. The first paper, 'A novel distributed air index for efficient spatial query processing in road sensor networks on the air' by Yanhong Li et al., explores the problem of spatial query processing in road sensor networks by means of wireless data broadcast. In addition it presents an efficient method to partition the record-keeping information about the underlying road sensor network and its associated objects, by which develops a fully distributed air index, called integrated exponential index, based on an extended version of the Hilbert curve. Besides, this paper also proposes efficient client-side algorithms to facilitate the processing of several kinds of spatial queries, including kNN query, CkNN query, and range query. The problem of enforcing the integrity of the outsourced data remotely is addressed in the paper, 'Parallel checking of content integrity in multi-cloud storage of heterogeneous sensor systems' by Jian Mao et al.. They propose a parallel cloud data possession checking scheme for the multi-cloud environment, which utilizes the homomorphic verification tag created by the Paillier cryptosystem to support unlimited query challenges and introduces the error-correction encoding method to ensure error localization and data correction. The paper, 'Subscribing to fuzzy temporal aggregation of heterogeneous sensor streams in real-time distributed environments' by J. Medina et al., presents an approach for distributing and processing heterogeneous data based on a representation with fuzzy linguistic terms, to solve the problems of the data fusion of sensors and the design of processing information. What's more, in order to illustrate the usefulness and effectiveness of this proposal, the authors present the results of the fuzzy temporal aggregation of sensor streams with alpha-cut subscriptions in a case study where an

SCIENTIFIC APPROACH TO SELECTION OF A HIGH-PERFORMANCE HETEROGENEOUS COMPUTING PLATFORM FOR IMAGE PROCESSING

The article substantiates the development of a scientific approach to the design of a high-performance heterogeneous computing platform (VGVP) for image processing. As an applied problem for realization on VGVP, the task of image processing in the regime of hard real time, coming through communication channels from diverse specialized information systems, was chosen. The considered scientific approaches realize the concept of creating a multi-module architecture, in which simultaneously several computational modules with a heterogeneous architecture can be processed by information flows. The article shows that heterogeneity, architectural solutions and program mechanisms of interaction of various technologies and technologies. VGVP for heterogeneous parallel-pipeline data processing.

Improved Semantic E-learning Framework for Complex Query Processing in Heterogeneous Web Services

Semantic e-learning has emerged as a next generation e-learning platform by enabling advanced manipulation, sharing, exchange, usage, and complete understanding of learning resources. Semantic framework is an appropriate methodology for extracting relevant contents frome-learning applications. However, Web-based data extraction suffers from the lack of discovery of heterogeneous Web services. Also, e-learning contents mainly focus on single domain ontologies. This paper presents an improved Semantic e-learning framework by extending or enhancing single domain ontologies to heterogeneous Web services. The proposed e-learning framework is built by integrating different and efficient matching mechanisms for handling and acquiring all kinds of heterogeneous Web-data contents and complex query data processing. The discovery of unified query processing services resulted in the extension of Web services. Our proposed framework also involves the processing of complex query in the heterogeneous e-learning framework. From the comparative analysis, it is clearly observed that the proposed framework yields higher precision, recall, and F-measure values than the existing attribute-based and item-based filtering algorithms.

Subscribing to fuzzy temporal aggregation of heterogeneous sensor streams in real‐time distributed environments

SummaryBecause of the deployment of heterogeneous sensors in intelligent environments, the fusion and information processing means an arduous and complex process. The data fusion of sensors and the design of processing information in real time are key aspects in order to generate feasible solutions. In order to shed light on this context, we present an approach for distributing and processing heterogeneous data based on a representation with fuzzy linguistic terms. In this way, the heterogeneous data from sensor streams are computed and summarized based on fuzzy temporal aggregations ubiquitously within mobile and ambient devices. This innovative approach provides an intuitive linguistic representation of mobile and ambient sensors as well as implies a drastic reduction of the communication burden. In order to provide high scalability in network communication, the information from sensor is spread under the publication‐subscription paradigm, where subscribers receive asynchronous events when the aggregation degree of the linguistic terms overcomes a threshold (alpha‐cut). Finally, in order to illustrate the usefulness and effectiveness of our proposal, we present the results of the fuzzy temporal aggregation of sensor streams with alpha‐cut subscriptions in a case study where an inhabitants performs an daily activities in an intelligent environment. Copyright © 2016 John Wiley & Sons, Ltd.

Modeling and interoperability of heterogeneous genomic big data for integrative processing and querying

While a huge amount of (epi)genomic data of multiple types is becoming available by using Next Generation Sequencing (NGS) technologies, the most important emerging problem is the so-called tertiary analysis, concerned with sense making, e.g., discovering how different (epi)genomic regions and their products interact and cooperate with each other. We propose a paradigm shift in tertiary analysis, based on the use of the Genomic Data Model (GDM), a simple data model which links genomic feature data to their associated experimental, biological and clinical metadata. GDM encompasses all the data formats which have been produced for feature extraction from (epi)genomic datasets. We specifically describe the mapping to GDM of SAM (Sequence Alignment/Map), VCF (Variant Call Format), NARROWPEAK (for called peaks produced by NGS ChIP-seq or DNase-seq methods), and BED (Browser Extensible Data) formats, but GDM supports as well all the formats describing experimental datasets (e.g., including copy number variations, DNA somatic mutations, or gene expressions) and annotations (e.g., regarding transcription start sites, genes, enhancers or CpG islands). We downloaded and integrated samples of all the above-mentioned data types and formats from multiple sources. The GDM is able to homogeneously describe semantically heterogeneous data and makes the ground for providing data interoperability, e.g., achieved through the GenoMetric Query Language (GMQL), a high-level, declarative query language for genomic big data. The combined use of the data model and the query language allows comprehensive processing of multiple heterogeneous data, and supports the development of domain-specific data-driven computations and bio-molecular knowledge discovery.

Heterogeneous Data Processing using Hadoop and Java Map/Reduce

Heterogeneous Data Processing using Hadoop and Java Map/Reduce

Methodological challenges and analytic opportunities for modeling and interpreting Big Healthcare Data.

Managing, processing and understanding big healthcare data is challenging, costly and demanding. Without a robust fundamental theory for representation, analysis and inference, a roadmap for uniform handling and analyzing of such complex data remains elusive. In this article, we outline various big data challenges, opportunities, modeling methods and software techniques for blending complex healthcare data, advanced analytic tools, and distributed scientific computing. Using imaging, genetic and healthcare data we provide examples of processing heterogeneous datasets using distributed cloud services, automated and semi-automated classification techniques, and open-science protocols. Despite substantial advances, new innovative technologies need to be developed that enhance, scale and optimize the management and processing of large, complex and heterogeneous data. Stakeholder investments in data acquisition, research and development, computational infrastructure and education will be critical to realize the huge potential of big data, to reap the expected information benefits and to build lasting knowledge assets. Multi-faceted proprietary, open-source, and community developments will be essential to enable broad, reliable, sustainable and efficient data-driven discovery and analytics. Big data will affect every sector of the economy and their hallmark will be ‘team science’.

A Stream Processing System for Multisource Heterogeneous Sensor Data

With the rapid development of the Internet of Things (IoT), a variety of sensor data are generated around everyone’s life. New research perspective regarding the streaming sensor data processing of the IoT has been raised as a hot research topic that is precisely the theme of this paper. Our study serves to provide guidance regarding the practical aspects of the IoT. Such guidance is rarely mentioned in the current research in which the focus has been more on theory and less on issues describing how to set up a practical system. In our study, we employ numerous open source projects to establish a distributed real time system to process streaming data of the IoT. Two urgent issues have been solved in our study that are (1) multisource heterogeneous sensor data integration and (2) processing streaming sensor data in real time manner with low latency. Furthermore, we set up a real time system to process streaming heterogeneous sensor data from multiple sources with low latency. Our tests are performed using field test data derived from environmental monitoring sensor data collected from indoor environment for system validation. The results show that our proposed system is valid and efficient for multisource heterogeneous sensor data integration and streaming data processing in real time manner.

The use of metaknowledge in a decision-making support system implemented in the functional neural network formalism

This paper describes the development of the functional-neural-network (FN-network) formalism, which was worked out to create a range of information systems for the intelligent computer processing of heterogeneous data from various information sources and automated decision-making support systems. It considers the limitations of this formalism in solving the tasks when the time intervals at which a solution is searched for are given in the form of indefinite variables. A method is proposed for avoiding these limitations by entering knowledge about the internal structure of data (metaknowledge) on these intervals into the context of a solution and it is shown that the use of metaknowledge not only solves the problem, but also improves the efficiency of searching for a solution by attracting additional information from a knowledge base and the context of a task.

A framework for mastering heterogeneity in multi-layer security information and event correlation

We detected limits of SIEM systems while being used to protect critical infrastructures from sophisticated cyberattacks.We developed a new data collection and pre-correlation framework named GET.GET links physical to logical security and exploits knowledge of the Business Process.The GET framework has been integrated into the open-source SIEM OSSIM.We validated the GET in a dam control system and a mobile phone based payment service. Security Information and Event Management (SIEM) is a consolidated technology that relies on the correlation of massive amounts of security-relevant information in order to detect ongoing attacks and intrusions. This correlation process is usually fed with logs generated by network devices and equipment, thus proving to be ineffective against attacks that affect multiple domains (e.g. physical, logical) or different architectural levels (e.g. network, operating system, application) of a service infrastructure. To bridge the gap, we propose a flexible framework for event collection and correlation, namely the Generic Event Translator, which is able to process heterogeneous data and spot evidence of security issues by using complex event pattern detectors that correlate information from multiple architectural layers and domains of the monitored infrastructure. The framework has been integrated into the open-source SIEM OSSIM, and validated in two challenging case studies, namely a dam infrastructure control system and a mobile phone based payment service. Display Omitted

The integrated environment selection and prototype сreation of thermal monitoring software

The components of information technology in the technological and technical auditing and monitoring of municipal buildings are investigated. The integrated environment is chosen and software prototype of subsystems of thermal monitoring software is created. The subsystem is aimed at the analysis and processing of heterogeneous data on energy efficiency of municipal property and performs decision support and cost calculations of energy-saving measures.The study was carried out as follows:1. The selected programming language is Java.2. The selected development environment is NetBeans. NetBeans IDE is integrated development environment written in Java. NetBeans program can run on platforms Windows, Linux, Mac OS and other platforms that support Java virtual machine (JVM), supports the development of programming languages Java, C/C++, PHP, JavaScript and others.3. The prototype of subsystems of thermal monitoring software is developed and described.These researches may help management of housing and communal services, economics and financial management of city councils to analyze and make ODA decisions to initiate energy saving projects.

Мethod of radiometric distortion correction of multispectral data for the earth remote sensing

The paper deals with technologies of ground secondary processing of heterogeneous multispectral data. The factors of heterogeneous data include uneven illumination of objects on the Earth surface caused by different properties of the relief. A procedure for the image restoration of spectral channels by means of terrain distortion compensation is developed. The object matter of this paper is to improve the quality of the results during image restoration of areas with large and medium landforms. Methods. Researches are based on the elements of the digital image processing theory, statistical processing of the observation results and the theory of multi-dimensional arrays. Main Results. The author has introduced operations on multidimensional arrays: concatenation and elementwise division. Extended model description for input data about the area is given. The model contains all necessary data for image restoration. Correction method for multispectral data radiometric distortions of the Earth remote sensing has been developed. The method consists of two phases: construction of empirical dependences for spectral reflectance on the relief properties and restoration of spectral images according to semiempirical data. Practical Relevance. Research novelty lies in development of the application theory of multidimensional arrays with respect to the processing of multispectral data, together with data on the topography and terrain objects. The results are usable for development of radiometric data correction tools. Processing is performed on the basis of a digital terrain model without carrying out ground works connected with research of the objects reflective properties.

Toward knowledge-based liver surgery: holistic information processing for surgical decision support.

Malignant neoplasms of the liver are among the most frequent cancers worldwide. Given the diversity of options for liver cancer therapy, the choice of treatment depends on various parameters including patient condition, tumor size and location, liver function, and previous interventions. To address this issue, we present the first approach to treatment strategy planning based on holistic processing of patient-individual data, practical knowledge (i.e., case knowledge), and factual knowledge (e.g., clinical guidelines and studies). The contributions of this paper are as follows: (1) a formalized dynamic patient model that incorporates all the heterogeneous data acquired for a specific patient in the whole course of disease treatment; (2) a concept for formalizing factual knowledge; and (3) a technical infrastructure that enables storing, accessing, and processing of heterogeneous data to support clinical decision making. Our patient model, which currently covers 602 patient-individual parameters, was successfully instantiated for 184 patients. It was sufficiently comprehensive to serve as the basis for the formalization of a total of 72 rules extracted from studies on patients with colorectal liver metastases or hepatocellular carcinoma. For a subset of 70 patients with these diagnoses, the system derived an average of [Formula: see text] assertions per patient. The proposed concept paves the way for holistic treatment strategy planning by enabling joint storing and processing of heterogeneous data from various information sources.

Guest editorial: selected papers from ICIMCS 2012

These papers cover key issues in Internet multimedia computing and services, including large-scale visual search, mobile media processing, Internet image tagging, video transmission, as well as fundamental technologies for intelligent multimedia computing, such as image segmentation, logo recognition, and object tracking. The first article “Visual word expansion and BSIFT verification for large-scale image search” by Zhou et al. presents a visual word expansion method improving the recall of retrieval on large-scale image collections and a binary SIFT verification method boosting the retrieval precision. The former represents a query image as the aggregation of visual words of local features in the query and the neighboring words of these query words. The latter verifies feature matching between images based on the binarized SIFT feature. The work is experimented on two public image data sets. The second article “Accurate sensing of scene geo-context via mobile visual localization” by Liu et al. proposes a new mobile image geo-tagging approach toward accurate location tagging for images captured in urban areas. For a landmark image, the approach infers comprehensive and accurate geo-context of the image, including camera location, view direction, and scene location, by exploiting large-scale image retrieval and 3D reconstruction techniques. Experimental results on the San Francisco street view data set, which consists of approximately 150 k panoramic images in San Francisco at about 4-m intervals, have demonstrated the effectiveness of the approach. The third article entitled “Tag ranking based on salient region graph propagation” by Tang et al. presents a new tag ranking solution to rank user-contributed tags associated with Internet images. The solution exploits salient regions in images and ranks tags by integrating visual clues from entire images and salient regions of images. It constructs two sparse graphs over images and salient sub-images, International Conference on Internet Multimedia Computing and Services (ICIMCS) is an annual conference sponsored by ACM SIGMM China Chapter. The conference is especially interested in the latest technologies and applications that deal with the Web-scale processing and management of heterogeneous data from the Internet for multimedia computing and service. ICIMCS 2012 was held in Wuhan, China. The conference attracted around 90 participants, including researchers from academia and industries across ten countries/regions, for sharing their recent works on the topics ranging from visual feature representation and analysis to visual recognition and classification, and from social and mobile media analysis to multimedia services. This special issue comprises the extended versions of seven papers, including three best papers and four papers from the regular and special sessions of ICIMCS 2012.