Big Data Analytics Tools Research Articles

Complex Power System Status Monitoring and Evaluation Using Big Data Platform and Machine Learning Algorithms: A Review and a Case Study

Efficient and valuable strategies provided by large amount of available data are urgently needed for a sustainable electricity system that includes smart grid technologies and very complex power system situations. Big Data technologies including Big Data management and utilization based on increasingly collected data from every component of the power grid are crucial for the successful deployment and monitoring of the system. This paper reviews the key technologies of Big Data management and intelligent machine learning methods for complex power systems. Based on a comprehensive study of power system and Big Data, several challenges are summarized to unlock the potential of Big Data technology in the application of smart grid. This paper proposed a modified and optimized structure of the Big Data processing platform according to the power data sources and different structures. Numerous open-sourced Big Data analytical tools and software are integrated as modules of the analytic engine, and self-developed advanced algorithms are also designed. The proposed framework comprises a data interface, a Big Data management, analytic engine as well as the applications, and display module. To fully investigate the proposed structure, three major applications are introduced: development of power grid topology and parallel computing using CIM files, high-efficiency load-shedding calculation, and power system transmission line tripping analysis using 3D visualization. The real-system cases demonstrate the effectiveness and great potential of the Big Data platform; therefore, data resources can achieve their full potential value for strategies and decision-making for smart grid. The proposed platform can provide a technical solution to the multidisciplinary cooperation of Big Data technology and smart grid monitoring.

Video Analytics for Railroad Safety Research: An Artificial Intelligence Approach

The volume of video data in the railroad industry has increased significantly in recent years. Surveillance cameras are situated on nearly every part of the railroad system, such as inside the cab, along the track, at grade crossings, and in stations. These camera systems are manually monitored, either live or subsequently reviewed in an archive, which requires an immense amount of human resources. To make the video analysis much less labor-intensive, this paper develops a framework for utilizing artificial intelligence (AI) technologies for the extraction of useful information from these big video datasets. This framework has been implemented based on the video data from one grade crossing in New Jersey. The AI algorithm can automatically detect unsafe trespassing of railroad tracks (called near-miss events in this paper). To date, the AI algorithm has analyzed hours of video data and correctly detected all near-misses. This pilot study indicates the promise of using AI for automated analysis of railroad video big data, thereby supporting data-driven railroad safety research. For practical use, our AI algorithm has been packaged into a computer-aided decision support tool (named AI-Grade) that outputs near-miss video clips based on user-provided raw video data. This paper and its sequent studies aim to provide the railroad industry with next-generation big data analysis methods and tools for quickly and reliably processing large volumes of video data in order to better understand human factors in railroad safety research.

Big data academic and learning analytics

PurposeAlthough big data analytics (BDA) have great benefits for higher education institutions (HEIs), due to lack of sufficient evidence on how BDA investment can pay off, it is tough for HEIs practitioners to realize value from such adoption. The purpose of this paper is to propose a big data academic and learning analytics enabled business value model to explain BDA potential benefits and business value which can be obtained by developing such analytics capabilities in HEIs.Design/methodology/approachThe study examined 47 case descriptions from 26 HEIs to investigate the causal association between the BDA current and potential benefits and business value creation path for big data academic and learning analytics success in HEIs.FindingsThe pressure of compliance with all legal and regulatory requirements and competition had pushed HEIs hard to adopt BDA tools. However, the study found out that application of risk and security and predictive analytics to higher education fields is still in its infancy. Using this theoretical model, the results provide new insights to higher education administrators on ways to create BDA capabilities for HEIs transformation and suggest an empirical foundation that can lead to more thorough analysis of BDA implementation.Originality/valueA distinctive theoretical contribution of this study is its conceptualization of understanding business value from BDA in the typical setting of higher education. The study provides HEIs with an all-inclusive understanding of BDA and gives insights on how it helps to transform HEIs. The new perspectives associated with the big data academic and learning analytics enabled business value model will contribute to future research in this area.

Predicting customer churn using targeted proactive retention

With the advent of innovative technologies and fierce competition, the choices for customers to choose from have increased tremendously in number. Especially in the case of a telecommunication industry, where deregulation is at its peak. Every year a new company springs up offering fitter options for its customers. This has turned the concentration of the business doers on churn prediction and business management models to sustain their places. Businesses approach churn in two ways, one is through targeted customer retention and through cause identification strategy. The literature of this paper provides a comprehensible understanding of the so far employed techniques in predicting customer churn. From that, it is quite evident that less attention has been given to the accuracy and the intuitiveness of churn models developed. Therefore, a novel approach of combining the models of Machine Learning and Big Data Analytics tools was proposed to deal churn prediction effectively. The purpose of this proposed work is to apply a novel retention technique called the targeted proactive retention to predict customer churning behavior in advance and help in their retention. This proposed work will help telecom companies to comprehend the risk associated with customer churn by predicting the possibility and the time of occurrence.

Big data and machine learning in critical care: Opportunities for collaborative research.

The introduction of clinical information systems (CIS) in Intensive Care Units (ICUs) offers the possibility of storing a huge amount of machine-ready clinical data that can be used to improve patient outcomes and the allocation of resources, as well as suggest topics for randomized clinical trials. Clinicians, however, usually lack the necessary training for the analysis of large databases. In addition, there are issues referred to patient privacy and consent, and data quality. Multidisciplinary collaboration among clinicians, data engineers, machine-learning experts, statisticians, epidemiologists and other information scientists may overcome these problems. A multidisciplinary event (Critical Care Datathon) was held in Madrid (Spain) from 1 to 3 December 2017. Under the auspices of the Spanish Critical Care Society (SEMICYUC), the event was organized by the Massachusetts Institute of Technology (MIT) Critical Data Group (Cambridge, MA, USA), the Innovation Unit and Critical Care Department of San Carlos Clinic Hospital, and the Life Supporting Technologies group of Madrid Polytechnic University. After presentations referred to big data in the critical care environment, clinicians, data scientists and other health data science enthusiasts and lawyers worked in collaboration using an anonymized database (MIMIC III). Eight groups were formed to answer different clinical research questions elaborated prior to the meeting. The event produced analyses for the questions posed and outlined several future clinical research opportunities. Foundations were laid to enable future use of ICU databases in Spain, and a timeline was established for future meetings, as an example of how big data analysis tools have tremendous potential in our field.

Digitalisation and Big Data Mining in Banking

Banking as a data intensive subject has been progressing continuously under the promoting influences of the era of big data. Exploring the advanced big data analytic tools like Data Mining (DM) techniques is key for the banking sector, which aims to reveal valuable information from the overwhelming volume of data and achieve better strategic management and customer satisfaction. In order to provide sound direction for the future research and development, a comprehensive and most up to date review of the current research status of DM in banking will be extremely beneficial. Since existing reviews only cover the applications until 2013, this paper aims to fill this research gap and presents the significant progressions and most recent DM implementations in banking post 2013. By collecting and analyzing the trends of research focus, data resources, technological aids, and data analytical tools, this paper contributes to bringing valuable insights with regard to the future developments of both DM and the banking sector along with a comprehensive one stop reference table. Moreover, we identify the key obstacles and present a summary for all interested parties that are facing the challenges of big data.

An Investigation Into the Efficacy of Deep Learning Tools for Big Data Analysis in Health Care

This article describes how machine learning (ML) algorithms are very useful for analysis of data and finding some meaningful information out of them, which could be used in various other applications. In the last few years, an explosive growth has been seen in the dimension and structure of data. There are several difficulties faced by conventional ML algorithms while dealing with such highly voluminous and unstructured big data. The modern ML tools are designed and used to deal with all sorts of complexities of data. Deep learning (DL) is one of the modern ML tools which are commonly used to find the hidden structure and cohesion among these large data sets by giving proper training in parallel platforms with intelligent optimization techniques to further analyze and interpret the data for future prediction and classification. This article focuses on the use of DL tools and software which are used in past couple of years in various areas and especially in the area of healthcare applications.

Applications of statistical techniques and artificial neural networks: A review

Big data is everywhere, and storage is affordable. The existing hardware and software are unable to analysis the vast amount of various types of data. Big data has become too complex and too dynamic to be able to process, store, analyze and manage with traditional data tools. Various statistical methods are important tools for big data analysis, but certain assumptions are required. So, a need emerges to explore an intelligent technique to analysis a large amount of data efficiently. One of the important approach used in machine learning is an artificial neural network (ANN) and is used to analysis the big data by using algorithms which allows learning, and the target value are based on data alone as compared to set of certain assumptions in statistical methods. Learning process can be simplified and automated by using iterative approach. This paper gives useful insight into the capabilities of Statistical Techniques and Artificial Neural Networks used in different types of applications.

Big data analytics tools a review

Big data is the hottest trending term all over the globe and the internet. Big organizations are trying to make use of the large amounts of data collected and stored by them in big memory storages. Further large amounts of data is being produced every millisecond all over the world from users of computing devices, from satellites of all kinds, from scientific research, from governments, from big organizations that deal with huge number of customers especially financial institutions and many more. These data lie there for exploration and exploitation to gain more knowledge or rather intelligence and turning out them into wisdom for better decision making. Traditional data mining tools are not able to handle this big data. Hadoop and MapReduce are the first of the kind of tools that are being used to handle big data. Additional data mining and machine learning capabilities have been added to Hadoop and MapReduce through various plug-ins by different open source as well as vendor tools for big data analytics (BDA). Further big organizations have and are in the process of creating BDA tools most of which come with a price tag. This study gives a short review of the available BDA tools taking into consideration different characteristics of these tools. Possible solutions for existing challenges related to big data analytics are discussed.

M-Health 2.0: New perspectives on mobile health, machine learning and big data analytics.

Mobile health (m-Health) has been repeatedly called the biggest technological breakthrough of our modern times. Similarly, the concept of big data in the context of healthcare is considered one of the transformative drivers for intelligent healthcare delivery systems. In recent years, big data has become increasingly synonymous with mobile health, however key challenges of 'Big Data and mobile health', remain largely untackled. This is becoming particularly important with the continued deluge of the structured and unstructured data sets generated on daily basis from the proliferation of mobile health applications within different healthcare systems and products globally. The aim of this paper is of twofold. First we present the relevant big data issues from the mobile health (m-Health) perspective. In particular we discuss these issues from the technological areas and building blocks (communications, sensors and computing) of mobile health and the newly defined (m-Health 2.0) concept. The second objective is to present the relevant rapprochement issues of big m-Health data analytics with m-Health. Further, we also present the current and future roles of machine and deep learning within the current smart phone centric m-health model. The critical balance between these two important areas will depend on how different stakeholder from patients, clinicians, healthcare providers, medical and m-health market businesses and regulators will perceive these developments. These new perspectives are essential for better understanding the fine balance between the new insights of how intelligent and connected the future mobile health systems will look like and the inherent risks and clinical complexities associated with the big data sets and analytical tools used in these systems. These topics will be subject for extensive work and investigations in the foreseeable future for the areas of data analytics, computational and artificial intelligence methods applied for mobile health.

Financial management and forecasting using business intelligence and big data analytic tools

This paper discusses about the latest and efficient financial tools and techniques which optimize the financial cost of the organization and predict the financial situation of the organization to flourish the business in the efficient way. Financial management basically means to accumulate funding for the enterprise at a low cost and to expend this collected funding for earning extreme profits. Thus, financial management means to plan and control the finance of the company. Financial management is usually concerned with the flow and control of money within an organization and be it either reserved or open sector. So, these tools if functional in the better way then business will reach in the utmost altitude. Recently some computational tools have been developed by the computer scientist for the efficient management and prediction for the business which is very useful for forecast and prediction in the era of digital world. The intention of this paper is to review and discuss the most significant applications of Business Intelligence in financial management and forecasting and prediction domain as well as point to new technology trends that will affect financial development.

Using multidimensional topological data analysis to identify traits of hip osteoarthritis.

Osteoarthritis (OA) is a multifaceted disease with many variables affecting diagnosis and progression. Topological data analysis (TDA) is a state-of-the-art big data analytics tool that can combine all variables into multidimensional space. TDA is used to simultaneously analyze imaging and gait analysis techniques. To identify biochemical and biomechanical biomarkers able to classify different disease progression phenotypes in subjects with and without radiographic signs of hip OA. Longitudinal study for comparison of progressive and nonprogressive subjects. In all, 102 subjects with and without radiographic signs of hip osteoarthritis. 3T, SPGR 3D MAPSS T1ρ /T2 , intermediate-weighted fat-suppressed fast spin-echo (FSE). Multidimensional data analysis including cartilage composition, bone shape, Kellgren-Lawrence (KL) classification of osteoarthritis, scoring hip osteoarthritis with MRI (SHOMRI), hip disability and osteoarthritis outcome score (HOOS). Analysis done using TDA, Kolmogorov-Smirnov (KS) testing, and Benjamini-Hochberg to rank P-value results to correct for multiple comparisons. Subjects in the later stages of the disease had an increased SHOMRI score (P < 0.0001), increased KL (P = 0.0012), and older age (P < 0.0001). Subjects in the healthier group showed intact cartilage and less pain. Subjects found between these two groups had a range of symptoms. Analysis of this subgroup identified knee biomechanics (P < 0.0001) as an initial marker of the disease that is noticeable before the morphological progression and degeneration. Further analysis of an OA subgroup with femoroacetabular impingement (FAI) showed anterior labral tears to be the most significant marker (P = 0.0017) between those FAI subjects with and without OA symptoms. The data-driven analysis obtained with TDA proposes new phenotypes of these subjects that partially overlap with the radiographic-based classical disease status classification and also shows the potential for further examination of an early onset biomechanical intervention. 2 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2018;48:1046-1058.

Big data analytics: Computational intelligence techniques and application areas

Big Data has significant impact in developing functional smart cities and supporting modern societies. In this paper, we investigate the importance of Big Data in modern life and economy, and discuss challenges arising from Big Data utilization. Different computational intelligence techniques have been considered as tools for Big Data analytics. We also explore the powerful combination of Big Data and Computational Intelligence (CI) and identify a number of areas, where novel applications in real world smart city problems can be developed by utilizing these powerful tools and techniques. We present a case study for intelligent transportation in the context of a smart city, and a novel data modelling methodology based on a biologically inspired universal generative modelling approach called Hierarchical Spatial-Temporal State Machine (HSTSM). We further discuss various implications of policy, protection, valuation and commercialization related to Big Data, its applications and deployment.

Barriers to big data analytics in manufacturing supply chains: A case study from Bangladesh

Recently, big data (BD) has attracted researchers and practitioners due to its potential usefulness in decision-making processes. Big data analytics (BDA) is becoming increasingly popular among manufacturing companies as it helps gain insights and make decisions based on BD. However, there many barriers to the adoption of BDA in manufacturing supply chains. It is therefore necessary for manufacturing companies to identify and examine the nature of each barrier. Previous studies have mostly built conceptual frameworks for BDA in a given situation and have ignored examining the nature of the barriers to BDA. Due to the significance of both BD and BDA, this research aims to identify and examine the critical barriers to the adoption of BDA in manufacturing supply chains in the context of Bangladesh. This research explores the existing body of knowledge by examining these barriers using a Delphi-based analytic hierarchy process (AHP). Data were obtained from five Bangladeshi manufacturing companies. The findings of this research are as follows: (i) data-related barriers are most important, (ii) technology-related barriers are second, and (iii) the five most important components of these barriers are (a) lack of infrastructure, (b) complexity of data integration, (c) data privacy, (d) lack of availability of BDA tools and (e) high cost of investment. The findings can assist industrial managers to understand the actual nature of the barriers and potential benefits of using BDA and to make policy regarding BDA adoption in manufacturing supply chains. A sensitivity analysis was carried out to justify the robustness of the barrier rankings.

Big Data in the Maritime Industry

The maritime industry is a complex system that requires quick adaptation to changing conditions and in which decision-making needs to take into account a large number of parameters. As navigation systems become more advanced, there is a significant amount of ship performance and navigation data generated. Big Data analytics tools make it possible to analyze these large quantities of data in order to gain insight that supports decision-making. This paper gives an overview of applications of Big Data in the maritime industry, specifically in logistics optimization, safety and energy efficiency improvement, as well as the challenges that systems involving Big Data face.

Challenges in Governing the Digital Transportation Ecosystem in Jakarta: A Research Direction in Smart City Frameworks

Mobility is one of the most difficult domains of the smart city to face. In fact, most large cities in the world are still facing urban mobility problems, especially traffic congestion. Particularly, in Jakarta, Indonesia, traffic congestion is a major issue that negatively affects productivity and the overall living quality of the citizens. Along with the development of the information communication and technology (ICT), the transportation domain in Jakarta has formed a digital transportation ecosystem, shown by the emergence of innovative digital-based transportation services. In line with this current condition, this paper hopes to contribute to the improvement of urban traffic in Jakarta by proposing research directions to govern the digital transportation ecosystem within a smart city framework. The significance of the research directions is reviewed using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology in a systematic review of previous studies. Ultimately, the research directions proposed in this paper lead to the necessity for an architectural perspective and relevant big data analytical tools to improve the digital transportation ecosystem in Jakarta.

On the blind use of statistical tools in the analysis of globular cluster stars

As with most data analysis methods, the Bayesian method must be handled with care. We show that its application to determine stellar evolution parameters within globular clusters can lead to paradoxical results if used without the necessary precautions. This is a cautionary tale on the use of statistical tools for big data analysis.

No One Owns Data

Businesses, policy makers, and scholars are calling for property rights in data. They currently focus particularly on the vast amounts of data generated by connected cars, industrial machines, artificial intelligence, toys and other devices on the Internet of Things (IoT). This data is personal to numerous parties who are associated with a connected device, for example, the driver of a connected car, its owner and passengers, as well as other traffic participants. Manufacturers, dealers, independent providers of auto parts and services, insurance companies, law enforcement agencies and many others are also interested in this data. Various parties are actively staking their claims to data on the Internet of Things, as they are mining data, the fuel of the digital economy. Stakeholders in digital markets often frame claims, negotiations and controversies regarding data access as one of ownership. Businesses regularly assert and demand that they own data. Individual data subjects also assume that they own data about themselves. Policy makers and scholars focus on how to redistribute ownership rights to data. Yet, upon closer review, it is very questionable whether data is—or should be—subject to any property rights. This article unambiguously answers the question in the negative, both with respect to existing law and future lawmaking, in the United States as in the European Union, jurisdictions with notably divergent attitudes to privacy, property and individual freedoms. Data as such, i.e., the content of information - exists conceptually separate from works of authorship and data bases (which can be subject to intellectual property rights), physical embodiments of information (data on a computer chip, which can be subject to personal property rights; warning symbol painted on a road, which can be subject to real property rights) and physical objects or intangible items to which information relates (e.g., a dangerous malfunctioning vehicle to which the warnings on road markings or a computer chip relate). Lawmakers have granted property rights to different persons regarding works of authorship, data bases, chattels, land and other items for the purpose of incentivizing investments and improvements in such items, a purpose that does not exist with respect to data as such. Individual persons, businesses, governments and the public at large have different interests in data and access restrictions. These interests are protected by an intricate net of existing laws that deliberately refrain from granting property laws in data. Existing property laws intentionally exclude data from subject matter definitions. Existing data-related laws and property laws balance interests in data and access restrictions based on public policy considerations that would be impaired by a creation of property rights in data. New property rights in data are not suited to promote better privacy or more innovation or technological advances, but would more likely suffocate free speech, information freedom, science and technological progress. The rationales for propertizing data are thus not compelling and are outweighed by the rationales for keeping the data “open.” No new property rights need to be created for data. The article begins with a brief review of the current landscape of the Internet of Things notes explosive growth of data pools generated by connected devices, artificial intelligence, big data analytics tools and other information technologies. Part 1 lays the foundation for examining concrete current legal and policy challenges in the remainder of the article. Part 2 supplies conceptual differentiation and definitions with respect to “data” and “information” as the subject of rights and interests. Distinctions and definitional clarity serve as the basis for examining the purposes and reach of existing property laws in Part 3, including real property, personal property and intellectual property laws. Part 4 analyzes the effect of data-related laws that do not grant property rights. Part 5 examines how the interests of the various stakeholders are protected or impaired by the current framework of data-related laws to identify potential gaps that could warrant additional property rights. Part 6 examines policy considerations for and against property rights in data. Part 7 concludes that no one owns data and no one should own data.

A survey towards an integration of big data analytics to big insights for value-creation

Big Data Analytics (BDA) is increasingly becoming a trending practice that generates an enormous amount of data and provides a new opportunity that is helpful in relevant decision-making. The developments in Big Data Analytics provide a new paradigm and solutions for big data sources, storage, and advanced analytics. The BDA provide a nuanced view of big data development, and insights on how it can truly create value for firm and customer. This article presents a comprehensive, well-informed examination, and realistic analysis of deploying big data analytics successfully in companies. It provides an overview of the architecture of BDA including six components, namely: (i) data generation, (ii) data acquisition, (iii) data storage, (iv) advanced data analytics, (v) data visualization, and (vi) decision-making for value-creation. In this paper, seven V's characteristics of BDA namely Volume, Velocity, Variety, Valence, Veracity, Variability, and Value are explored. The various big data analytics tools, techniques and technologies have been described. Furthermore, it presents a methodical analysis for the usage of Big Data Analytics in various applications such as agriculture, healthcare, cyber security, and smart city. This paper also highlights the previous research, challenges, current status, and future directions of big data analytics for various application platforms. This overview highlights three issues, namely (i) concepts, characteristics and processing paradigms of Big Data Analytics; (ii) the state-of-the-art framework for decision-making in BDA for companies to insight value-creation; and (iii) the current challenges of Big Data Analytics as well as possible future directions.

Digital revolution and Big Data: a new revolution in agriculture.

Abstract This review considers the role of Big Data (BD), the digital revolution, the application of Internet of Things (loTs) and sensor technologies in the agriculture sector. The introduction is focussed on the ongoing research efforts on BD within agriculture sector, basic features of BD and latest development in BD analytics tools. In subsequent sections, the importance of BD applications in the agriculture sector and examples of their success stories in increasing farm productivity, current scenario on BD and digital agriculture, the future prospects of BD and bottlenecks in its implementation in agriculture sector are discussed. Agriculture sector is undergoing a new revolution and transformation, driven by IoT, sensor technologies, BD and cloud computing. This digital revolution in agriculture is very promising and will enable the agriculture sector to move to the next level of farm productivity and profitability. This transformation process looks irreversible and poised to revolutionize not only agriculture but the entire farm-to-food sector.