Big Data Solutions Research Articles

Applications of the MapReduce programming framework to clinical big data analysis: current landscape and future trends.

The emergence of massive datasets in a clinical setting presents both challenges and opportunities in data storage and analysis. This so called “big data” challenges traditional analytic tools and will increasingly require novel solutions adapted from other fields. Advances in information and communication technology present the most viable solutions to big data analysis in terms of efficiency and scalability. It is vital those big data solutions are multithreaded and that data access approaches be precisely tailored to large volumes of semi-structured/unstructured data.The MapReduce programming framework uses two tasks common in functional programming: Map and Reduce. MapReduce is a new parallel processing framework and Hadoop is its open-source implementation on a single computing node or on clusters. Compared with existing parallel processing paradigms (e.g. grid computing and graphical processing unit (GPU)), MapReduce and Hadoop have two advantages: 1) fault-tolerant storage resulting in reliable data processing by replicating the computing tasks, and cloning the data chunks on different computing nodes across the computing cluster; 2) high-throughput data processing via a batch processing framework and the Hadoop distributed file system (HDFS). Data are stored in the HDFS and made available to the slave nodes for computation.In this paper, we review the existing applications of the MapReduce programming framework and its implementation platform Hadoop in clinical big data and related medical health informatics fields. The usage of MapReduce and Hadoop on a distributed system represents a significant advance in clinical big data processing and utilization, and opens up new opportunities in the emerging era of big data analytics. The objective of this paper is to summarize the state-of-the-art efforts in clinical big data analytics and highlight what might be needed to enhance the outcomes of clinical big data analytics tools. This paper is concluded by summarizing the potential usage of the MapReduce programming framework and Hadoop platform to process huge volumes of clinical data in medical health informatics related fields.

Breaking the chains

Data management research, systems, and technologies have drastically improved the availability of data analysis capabilities, particularly for non-experts, due in part to low-entry barriers and reduced ownership costs (e.g., for data management infrastructures and applications). Major reasons for the widespread success of database systems and today's multi-billion dollar data management market include data independence , separating physical representation and storage from the actual information, and declarative languages , separating the program specification from its intended execution environment. In contrast, today's big data solutions do not offer data independence and declarative specification. As a result, big data technologies are mostly employed in newly-established companies with IT-savvy employees or in large well-established companies with big IT departments. We argue that current big data solutions will continue to fall short of widespread adoption, due to usability problems, despite the fact that in-situ data analytics technologies achieve a good degree of schema independence. In particular, we consider the lack of a declarative specification to be a major road-block, contributing to the scarcity in available data scientists available and limiting the application of big data to the IT-savvy industries. In particular, data scientists currently have to spend a lot of time on tuning their data analysis programs for specific data characteristics and a specific execution environment. We believe that the research community needs to bring the powerful concepts of declarative specification to current data analysis systems, in order to achieve the broad big data technology adoption and effectively deliver the promise that novel big data technologies offer.

Big Data in Healthcare - Defining the Digital Persona through User Contexts from the Micro to the Macro. Contribution of the IMIA Organizational and Social Issues WG.

While big data offers enormous potential for improving healthcare delivery, many of the existing claims concerning big data in healthcare are based on anecdotal reports and theoretical vision papers, rather than scientific evidence based on empirical research. Historically, the implementation of health information technology has resulted in unintended consequences at the individual, organizational and social levels, but these unintended consequences of collecting data have remained unaddressed in the literature on big data. The objective of this paper is to provide insights into big data from the perspective of people, social and organizational considerations. We draw upon the concept of persona to define the digital persona as the intersection of data, tasks and context for different user groups. We then describe how the digital persona can serve as a framework to understanding sociotechnical considerations of big data implementation. We then discuss the digital persona in the context of micro, meso and macro user groups across the 3 Vs of big data. We provide insights into the potential benefits and challenges of applying big data approaches to healthcare as well as how to position these approaches to achieve health system objectives such as patient safety or patient-engaged care delivery. We also provide a framework for defining the digital persona at a micro, meso and macro level to help understand the user contexts of big data solutions. While big data provides great potential for improving healthcare delivery, it is essential that we consider the individual, social and organizational contexts of data use when implementing big data solutions.

EHR Big Data Deep Phenotyping. Contribution of the IMIA Genomic Medicine Working Group.

Given the quickening speed of discovery of variant disease drivers from combined patient genotype and phenotype data, the objective is to provide methodology using big data technology to support the definition of deep phenotypes in medical records. As the vast stores of genomic information increase with next generation sequencing, the importance of deep phenotyping increases. The growth of genomic data and adoption of Electronic Health Records (EHR) in medicine provides a unique opportunity to integrate phenotype and genotype data into medical records. The method by which collections of clinical findings and other health related data are leveraged to form meaningful phenotypes is an active area of research. Longitudinal data stored in EHRs provide a wealth of information that can be used to construct phenotypes of patients. We focus on a practical problem around data integration for deep phenotype identification within EHR data. The use of big data approaches are described that enable scalable markup of EHR events that can be used for semantic and temporal similarity analysis to support the identification of phenotype and genotype relationships. Stead and colleagues' 2005 concept of using light standards to increase the productivity of software systems by riding on the wave of hardware/processing power is described as a harbinger for designing future healthcare systems. The big data solution, using flexible markup, provides a route to improved utilization of processing power for organizing patient records in genotype and phenotype research.

Big Data solutions on a small scale: Evaluating accessible high-performance computing for social research

Though full of promise, Big Data research success is often contingent on access to the newest, most advanced, and often expensive hardware systems and the expertise needed to build and implement such systems. As a result, the accessibility of the growing number of Big Data-capable technology solutions has often been the preserve of business analytics. Pay as you store/process services like Amazon Web Services have opened up possibilities for smaller scale Big Data projects. There is high demand for this type of research in the digital humanities and digital sociology, for example. However, scholars are increasingly finding themselves at a disadvantage as available data sets of interest continue to grow in size and complexity. Without a large amount of funding or the ability to form interdisciplinary partnerships, only a select few find themselves in the position to successfully engage Big Data. This article identifies several notable and popular Big Data technologies typically implemented using large and extremely powerful cloud-based systems and investigates the feasibility and utility of development of Big Data analytics systems implemented using low-cost commodity hardware in basic and easily maintainable configurations for use within academic social research. Through our investigation and experimental case study (in the growing field of social Twitter analytics), we found that not only are solutions like Cloudera’s Hadoop feasible, but that they can also enable robust, deep, and fruitful research outcomes in a variety of use-case scenarios across the disciplines.

New Design Principles for Effective Knowledge Discovery from Big Data

Big data is creating hype in IT industry. Knowledge discovery from big data can allow organizations to have deeper insights, look at the bigger picture and project big returns. There are various principles that have been presented for knowledge discovery from big data by ORNL (Oak Ridge National University), USA. These are: (i) support a variety of analysis methods, (ii) one size doesn’t fit all, (iii) make data accessible. However timeliness and security still pose great challenges in the knowledge discovery process. Timely analysis of big data is essential because data is being produced at a very high velocity. Security of big data is difficult to ensure since big data solutions were not developed with security in mind. In this paper, we give a view of various big data dimensions and present two new principles based on security and timely analysis for knowledge discovery from big data.

Obtain confidentiality or/and authenticity in Big Data by ID-based generalized signcryption

Recently, the Big Data paradigm has received considerable attention since it gives a great opportunity to mine knowledge from massive amounts of data. However, the new mined knowledge will be useless if data is fake, or sometimes the massive amounts of data cannot be collected due to the worry on the abuse of data. This situation asks for new security solutions. On the other hand, the biggest feature of Big Data is “massive”, which requires that any security solution for Big Data should be “efficient”. In this paper, we propose a new identity-based generalized signcryption scheme to solve the above problems. In particular, it has the following two properties to fit the efficiency requirement. (1) It can work as an encryption scheme, a signature scheme or a signcryption scheme as per need. (2) It does not have the heavy burden on the complicated certificate management as the traditional cryptographic schemes. Furthermore, our proposed scheme can be proven-secure in the standard model.

A Dashboard of an Education Data Portal using Big Data Solutions

Educational Data Portal (EDP) play important role in teaching and learning as it contains useful resources. Every big educational institutions such as university shall build an EDP soon or later. The aim of this study is to utilize Big Data solutions in building a Dashboard for an Education Data Portal. The proposed EDP is envisioned to be a core tool for all students and learning agencies, providing support for many types of views and content/instructional resources to allow effective data-driven decision-making for students, teacher and the public, based on recent standards. It supports many features such as accessibility of data and content anywhere, scalability, extensibility of functionality, and extensibility of the technology architecture to support integration with the Shared Learning Infrastructure (SLI). The Data Dashboard is highly scalable and extensible architecture that will grow, if necessary, to meet the needs of students, and educators

Environmental Conditions’ Big Data Management and Cloud Computing Analytics for Sustainable Agriculture

The World is getting swamped with data to the tune of 7ZB a year mostly emanating from the 'internet of things' devices. This data is scattered in various devices and no meaningful relationship may be derived from it, and neither can this terrific rate be managed by traditional storage or processors. It is believed that organizations that are best able to make real-time business decisions using Big Data solutions will thrive, while those that are unable to embrace and make use of this shift will increasingly find themselves at a competitive disadvantage in the market and face potential failure. Cloud computing promises enough capacity in terms of storage and processing power to elastically handle data of such magnitude and through the use of analytics get value from it. This paper focuses on environmental conditions' data like rainfall, winds, temperature etc and the use of particular cloud computing analytical tool to get some meaningful information from it which can be utilized by farmers for strategic and successful Agriculture. Previous similar studies are discussed and recommendations given.

Toward Scalable Systems for Big Data Analytics: A Technology Tutorial

Recent technological advancements have led to a deluge of data from distinctive domains (e.g., health care and scientific sensors, user-generated data, Internet and financial companies, and supply chain systems) over the past two decades. The term big data was coined to capture the meaning of this emerging trend. In addition to its sheer volume, big data also exhibits other unique characteristics as compared with traditional data. For instance, big data is commonly unstructured and require more real-time analysis. This development calls for new system architectures for data acquisition, transmission, storage, and large-scale data processing mechanisms. In this paper, we present a literature survey and system tutorial for big data analytics platforms, aiming to provide an overall picture for nonexpert readers and instill a do-it-yourself spirit for advanced audiences to customize their own big-data solutions. First, we present the definition of big data and discuss big data challenges. Next, we present a systematic framework to decompose big data systems into four sequential modules, namely data generation, data acquisition, data storage, and data analytics. These four modules form a big data value chain. Following that, we present a detailed survey of numerous approaches and mechanisms from research and industry communities. In addition, we present the prevalent Hadoop framework for addressing big data challenges. Finally, we outline several evaluation benchmarks and potential research directions for big data systems.

Study of CDR Real-Time Query Based on Big Data Technologies

This paper describes big data technology layers, analyses the CDR (Call Data Records) real-time query scenario of telecommunications and brings forward a fast indexing and query solution based on the open source Hadoop platform. A CDR real-time query system was built according to the solution. A performance test was conducted with the real dataset of a city with 3 million subscribers. Compared with the existing system, the big data solution can greatly improve data processing performance and support real-time query with lower hardware and software investment.

Core System Transformation and Big Data Re-Architecting

Core System Transformation and Big Data Re-Architecting

Global Trends in Medical Journal Publishing

Global Trends in Medical Journal Publishing

Detection of the onset of agitation in patients with dementia: real-time monitoring and the application of big-data solutions

The changing demographic profile of the population has potentially challenging consequences resulting in a rapidly growing elderly population with healthcare implications including Alzheimer type conditions (a leading cause of dementia). Dementia requires long term care to manage the negative behavioural symptoms which are primarily exhibited in terms of agitation and aggression. This article considers dementia with a discussion around independent assisted living (IAL). We discuss the behavioural and psychological symptoms of dementia (BPSD) and the factors (precursors) to the onset of agitation and aggression. Context and data processing is introduced and the nature of knowledge considered. Next generation context-aware systems are considered with the focus on health monitoring. We postulate that the challenges lie in the effective realisation of IAL; achieving this however remains an objective of intensive multi-disciplinary research involving both clinicians and computer science in the development of software and non-invasive sensor technologies implemented in mobile systems.