Big Data Tools Research Articles

Impact Analysis of COVID-19 Pandemic on Istanbul Traffic with Big Data Tools

Impact Analysis of COVID-19 Pandemic on Istanbul Traffic with Big Data Tools

Development of a web-based platform for the semi-structured record of the psychiatric interview during clinical practice: an opportunity to impact research and improve health care

IntroductionManaging healthcare data is a major challenge for today’s medicine. The use of artificial intelligence and big data tools has allowed solving questions related to this topic. However, the wide heterogeneity in the psychiatric consultation record makes the retrospective analysis of these data limited due to a lack of information or differences between specialists.ObjectivesWe aim to develop a platform that allows the structured record of medical care data (based on dementia) while maintaining flexibility and format for its usefulness during clinical practice in psychiatry.MethodsWe developed a web-based platform for the structured and semi-structured record of psychiatric evaluation. The instrument is diagnosed-oriented (for our version we used dementia). We used Core outcome sets and expert opinion to identify the relevant outcomes for the attention.ResultsA web-based platform is presented for the care of people with suspected dementia at different levels of care designed with the potential to record information of interest in research but also of clinical utility for closer follow-up.ConclusionsThis strategy allows developing the proposal towards other pathologies of interest. Also, with the integration of recommendation algorithms, a monitoring and recommendation system could be achieved to promote knowledge of psychiatric illness from routine practice. This proposal intends to have an impact by increasing the quality of care, reducing care times, and providing better approaches from primary care systems.DisclosureNo significant relationships.

Business Decision Making by Big Data Analytics

Information is the key component towards success when it comes to controlling the decision-makers performance with the quality of a decision. In the modern era, an absolute amount of data is available to organizations for analysis usage. Data is the most important component of the business in the 21st century and a significant number of devices are already equipped with the internet. Based on this the solutions should be studied in order to control and capture the knowledge value pair out of the datasets. Following this, the decision-makers should have access to insightful and valuable data based on the dynamic high volume & velocity using big data analytics. Our research focuses on how to integrate big data analytics into the decision-making process. The B-DAD (big data analytics and decision) framework was created to map the big data tools, its architecture, and analytics for the several decision-making steps by the adoption of methodology based on design science. The ideal goal and offerings of the framework are adopting big data analytics in order to intensify & aid decision making for the organization using an integration of big data analytics into the corresponding decision-making process. Thus, the experiment was carried out in the retail domain to test the framework. As an end result, the results showcased the value-added if big data analytics is integrated with corresponding decision-making activity.

Divergent Big Data Tools and Its applications in Different Domains

Divergent Big Data Tools and Its applications in Different Domains

Multidimensional architecture using a massive and heterogeneous data: Application to drought monitoring

The rapid increase in the number of Earth Observation (EO) systems generates a massive amount of heterogeneous data. It has raised big issues in collecting, preprocessing, storing, and the visualization these data. However, traditional techniques are facing serious challenges when dealing with big EO data dimensions (i.e., Volume, Veracity, Variety, and Velocity), especially in natural hazards management. Therefore, big data techniques and tools attract more attention. In this paper we propose a multidimensional model framework for Big EO data warehousing. This framework includes 3 parts: (1) Data collection and preprocessing, being responsible for collecting data and improving their quality; (2) Data loading and storage, performing the ingestion task which consists of transferring the data from external resources to the Big data platform for storage; and (3) Visualization and interpretation, aiming to provide spatio-temporal analysis. This framework could be useful for decision-makers in monitoring the effects of drought disasters and, consequently, planning the mitigation and remediation measures. Experiments are carried out on drought monitoring in China along the period 2000–2020. The input data include remote sensing data, biophysical data, and climatological data. The results reveal that the proposed framework has a higher retrieval speed and a greater elasticity with different kinds (i.e. spatial, temporal, or spatiotemporal) of requests compared to traditional frameworks, indicating its superiority.

An efficient system using implicit feedback and lifelong learning approach to improve recommendation

This paper presents a new technique for contextual item-to-item Collaborative Filtering-based Recommender System, an improved version popularised by e-commerce giant Amazon two decades back. The concept is based on items also-viewed under the same browsing session. Users’ browsing patterns, locations, and timestamps are considered as the context and latent factors for each user. The algorithm computes recommendations based on users’ implicit endorsements by clicks. The algorithm does not enforce the user to log in to provide recommendations and is capable of providing accurate recommendations for non-logged-in users and with a setting where the system is unaware of users’ preferences and profile data (non-logged-in users). This research takes the cue from human lifelong incremental learning experience applied to machine learning on a large volume of the data pool. First, all historical data is gathered from collectable sources in a distributed manner through big data tools. Then, a long-running batch job creates the initial model and saves it to Hadoop Distributed File System (HDFS). An ever-running streaming job loads the model from HDFS and builds on top of it in an incremental fashion. At the architectural level, this resembles the big data mix processing Lambda Architecture. The recommendation is computed based on a proposed equation for a weighted sum between near real-time and historical batch data. Real-time and batch processing engines act as autonomous Multi-agent systems in collaboration. We propose an ensemble method for batch-stream the recommendation engine. We introduce a novel Lifelong Learning Model for recommendation through Multi-agent Lambda Architecture. The recommender system incrementally updates its model on streaming datasets to improve over time.

A 25-Year Trend of Catastrophic Health Expenditure and Its Inequality in China: Evidence from Longitudinal Data.

PurposeThe Chinese government has carried out two major cycles of reform to improve the health system and reduce the disease burden on residents. This study aims to comprehensively track the trends in the occurrence of catastrophic health expenditure (CHE) and its inequality in the past 25 years, which may help better understand the influence of health system reforms on CHE and its inequality.MethodsThe study employed the subset of data from China Health and Nutrition Survey conducted from 1991 to 2015. Health payments and net household income were used to calculate CHE. Concentration index and decomposition analysis were used to measure the magnitude of income-related inequality in CHE and decompose it into determinants, respectively.ResultsThe incidence of CHE in China increased from 3.10% in 1993 to 8.90% in 2004 and still maintained at a high level in the following years. The incidence gap of CHE between the richest and poorest became increasingly wider over year. Moreover, the values of adjusted concentration indexes were all negative in each year, decreasing from −0.202 in 1991 to −0.613 in 2015. Income was consistently the largest contributor to the inequality in CHE. The basic medical insurance did not decrease the incidence of CHE and showed the second largest contribution on its inequality before 2004. However, this contribution began to decline after 2006.ConclusionAfter the New Health Care Reform, despite many measures taken by the Chinese government, there was still a high incidence of CHE and an increasing inequality from 1991 to 2015. The basic medical insurance in China was not enough to protect households from CHE. The use of big data tools and techniques to effectively screen the poor households, and strengthening the social medical aid system would be helpful to decrease the inequality in CHE.

The Shrewd Provoke of DATA Analytics in Automotive Industry

Big Data is taking the world by storm. With the huge amounts of data emanating from various digital sources the importance of analytics has tremendously grown making the companies to tap the dark data that was considered useless all these years (2). Big Data tools and Technologies help the companies to interpret the huge amount of data very faster which helps to boost production. So, Big data applications are creating a new era in every industry (3). Transportation is a vital part of society and our everyday lives, so ensuring that our drives are safe is a top priority. Advancement in technology such as phone apps, black boxes, smart cars and so on, are all designed to take the human error factor out of driving and should help to enhance ultimate road safety. The Proposed system focuses on novel approach to provide enhanced road safety mechanism using Data Analytics.

System Architecture of Big Data in Massive Open Online Courses (BD-MOOCs System Architecture)

The system architecture of big data in massive open online courses (BD-MOOCs System Architecture) is composed of six components. The first component was comprised of big data tools and technologies such as Hadoop, YARN, HDFS, Spark, Hive, Sqoop, and Flume. The second component was educational data science, which is composed of the following four parts: EDM, ERS, AA, and S/II. The third component was a description of three basic elements of a big data system: data capture, management, and analysis. The fourth component was that MOOCs were classified as cMOOCs, xMOOCs, quasi-MOOCs, hMOOCs, and other related. The fifth component included the steps of MOOC development: design, delivery, and assessment. Finally, MOOCs present educational data science challenges such as analyzing student interactions, estimating dropout risk, grading, and making recommendations. Overall, the BD-MOOCs system architecture design was suitable at the highest level.

USE OF INNOVATIVE INFORMATION TECHNOLOGY IN E-COMMERCE AND DIGITAL ECONOMY

The article considers the use of modern innovative information technology (IT) in general in the digital economy and in e-commerce, as its most sensitive part. First of all, the paper shows that since the beginning of the XXI century one of the main trends in the economy and society in general is the penetration of innovative information technologies in various fields of human activity, which entails global digitalization of both economy and society as a whole. E-commerce, as part of the digital economy, has been shown to be a driving force in accelerating global economic development and creating new markets. E-commerce also opens up new opportunities for inclusive and sustainable innovation growth. It is argued that the rapid development of e-commerce became possible only after the percentage of global Internet penetration exceeded some critical level, which led to its availability to the general public. In turn, the emergence of more and more new users on the Internet, the widespread use of various social networks and the rapid growth of internet of things (IoT) has led to the data explosion and the emergence and spread of technology such as Big Data. The main advantages of using Big Data in modern analytics in comparison with traditional methods are considered in the work. The place and role of Big Data technologies and tools in the digitalization of the Ukrainian economy are analyzed. It is proved that the intensive development and spread of the use of Big Data in all spheres of modern society leads to the active use of innovative IT. Based on the analysis, the main areas of use of Big Data are identified, namely: credit scoring, segmented or personalized services, risk management, anti-fraud system and using blockchain technology. The use of these technologies, in turn, will minimize customer effort and increase the efficiency of B2B and B2C models in both the digital economy and e-commerce.

Using and Processing Big Data in Transport – Platforms and Approaches

The article converges on the topics of extracting, communicating, processing, utilization, and analysis of big data in transportation. It looks at some aspects of the application of the Internet of Things and big data in the field of transport. The sources of big data in this sector are presented, depicting a general classification according to their place of origin. The main data communication transport protocols are shown, in addition to the different types of cloud platforms that can receive, store and process data. A brief description of the functions of transport control centers is made, pointing out some issues related to the extraction, storage and analysis of big data. At the end, a source of transportation data is used to verify the possibility for expanding the operation of a transport control center, which can work collaboratively with a proposed big data platform and extract, process and analyze data with appropriate open source tools for big data.

A human resources analytics and machine-learning examination of turnover: implications for theory and practice

PurposeWhat do antecedents of turnover tell us when examined using human resources (HR) analytics and machine-learning tools, and what are the respective theoretical and practical implications? Although the turnover literature is expansive, empirical evidence on turnover antecedents studied using data science tools remains limited.Design/methodology/approachTo help reinvigorate research in this field, the authors propose a novel examination of turnover antecedents—competencies, commitment, trust and cultural values—using big data tools to develop a granular, case-dependent measure of turnover.FindingsUsing archival data from 700,000 employees of a large organization collected over a period of ten years, the authors find that turnover is generally associated with varying levels of these antecedents. However, in more fine-grained analysis, their relation to turnover is contingent upon role, person and cultural background.Originality/valueThe authors discuss the implications on turnover and strategic HR research and the potential of Artificial Intelligence and machine-learning methods in the design and implementation of managerial and HR planning initiatives.

Twitter Sentiment Analysis for Reviewing Tourist Destinations in Saudi Arabia using Apache Spark and Machine Learning Algorithms

The appearance of big data has created new challenges for data analysis teams especially dealing with unstructured data in text form. Many applications increasingly include a large amount of this type of data. Example of such data is data collected from Twitter. Adequate use of Machine Learning (ML), big data tools and social media platforms can solve several problems. The aim of this research is to apply sentiment analysis using Arabic tweets of tourism in Saudi Arabia and determine the most visited places. Ara Senti corpus was used as the labelled data to perform machine learning for sentiment analysis to deal with the Arabic morphology. The three-classes classification (Positive, Negative, or Neutral) was performed using Decision Tree, Random Forest, Logistic Regression and Naïve Bayes. The results showed that the highest performance achieved was 86% using Logistic Regression with Term Frequency–Inverse Document Frequency (TF-IDF) representation and Naïve Bayes with Bag-of-Words model compared with both random forest and decision tree. The trainable classifier was applied to predict classes on collected data from Twitter for reviewing Kingdom of Saudi Arabia (KSA) destinations to finally present a rating of the most visited places on KSA. There are five most visited places in Saudi Arabia (Riyadh, Alula, Hail, Taif and Tabuk).

A Standardized GIS Web Service Oriented Shared-nothing Architecture - Taking Species Distribution Modeling as an Example

The rapid development of big data analytics technologies and tools, particularly the related distributed computing technologies, which divide large problems into smaller ones, provides important technical support for developing big earth data research. To realize the effective utilization of big data computing resources, it is important to overcome the potential incompatibility and inconsistency among the distributed computing nodes. Using standardized GIS web service interfaces is a promising approach to link existing heterogeneous GIS platforms through standardized protocols and establish a universal, platform-independent computing layer to improve the efficiency of computing resources for big earth data analysis. This research analyzed the requirements of distributed computing, and further proposed a standardized GIS web service-oriented shared-nothing architecture by comparing various mainstream distributed computing architectures and related web service specifications in the industry. For example, the prediction of Lophelia pertusa coral distribution by random forest is the application of the method discussed in this paper.

Use of Diagnosis Codes to Find Blood Transfusion Adverse Events in Electronic Health Records.

Electronic health records (EHRs) and big data tools offer the opportunity for surveillance of adverse events (patient harm associated with medical care). We used International Classification of Diseases, Ninth Revision, codes in electronic records to identify known, and potentially novel, adverse reactions to blood transfusion. We used 49,331 adult admissions involving critical care at a major teaching hospital, 2001-2012, in the Medical Information Mart for Intensive Care III EHRs database. We formed a T (defined as packed red blood cells, platelets, or plasma) group of 21,443 admissions versus 25,468 comparison (C) admissions. The International Classification of Diseases, Ninth Revision, Clinical Modification , diagnosis codes were compared for T versus C, described, and tested with statistical tools. Transfusion adverse events (TAEs) such as transfusion-associated circulatory overload (TACO; 12 T cases; rate ratio [RR], 15.61; 95% confidence interval [CI], 2.49-98) were found. There were also potential TAEs similar to TAEs, such as fluid overload disorder (361 T admissions; RR, 2.24; 95% CI, 1.88-2.65), similar to TACO. Some diagnoses could have been sequelae of TAEs, including nontraumatic compartment syndrome of abdomen (52 T cases; RR, 6.76; 95% CI, 3.40-14.9) possibly being a consequence of TACO. Surveillance for diagnosis codes that could be TAE sequelae or unrecognized TAE might be useful supplements to existing medical product adverse event programs.

Big Data for Traffic Estimation and Prediction: A Survey of Data and Tools

Big data have been used widely in many areas, including the transportation industry. Using various data sources, traffic states can be well estimated and further predicted to improve the overall operation efficiency. Combined with this trend, this study presents an up-to-date survey of open data and big data tools used for traffic estimation and prediction. Different data types are categorized, and off-the-shelf tools are introduced. To further promote the use of big data for traffic estimation and prediction tasks, challenges and future directions are given for future studies.

Intelligent Data Analysis for Infection Spread Prediction

Intelligent data analysis based on artificial intelligence and Big Data tools is widely used by the scientific community to overcome global challenges. One of these challenges is the worldwide coronavirus pandemic, which began in early 2020. Data science not only provides an opportunity to assess the impact caused by a pandemic, but also to predict the infection spread. In addition, the model expansion by economic, social, and infrastructural factors makes it possible to predict changes in all spheres of human activity in competitive epidemiological conditions. This article is devoted to the use of anonymized and personal data in predicting the coronavirus infection spread. The basic “Susceptible–Exposed–Infected–Recovered” model was extended by including a set of demographic, administrative, and social factors. The developed model is more predictive and applicable in assessing future pandemic impact. After a series of simulation experiment results, we concluded that personal data use in high-level modeling of the infection spread is excessive.

Internet of Things Data Visualization for Business Intelligence.

This study contributes to the research on Internet of Things data visualization for business intelligence processes, an area of growing interest to scholars, by conducting a systematic review of the literature. A total of 237 articles published over the past 11 years were obtained and compared. This made it possible to identify the top contributing and most influential authors, countries, publishers, institutions, papers, and research findings, together with the challenges facing current research. Based on these results, this work provides a thorough insight into the field by proposing four research categories (Technology infrastructure, Case examples, Final-user experience, and Big Data tools), together with the development of these research streams over time and their future research directions.

One size does not fit all: an historian's perspective on precision diabetes medicine.

This article offers an historical approach to exploring precision medicine's potential for reducing health disparities in diabetes. It examines case studies from the twentieth-century USA, from early twentieth-century beliefs that Jews were most at risk of developing diabetes to claims in the 1980s that Native Americans, African Americans, Hispanic Americans and Asian Americans had the greatest likelihood of developing the disease. These case studies reveal that attempts to understand perceived health disparities have long tended to focus on the biology and behaviours of the unwell, while paying less attention to food security, workplace hazards, access to quality healthcare and other social determinants of health. The precision medicine initiative, I argue, has an opportunity to right this imbalance by leveraging the tools of big data to learn more not only about biomarkers but also about the social and physical environments in which people live and work.

Prediction of Oil Production through Linear Regression Model and Big Data Tools

Fossil fuels, including oil, are the most important sources of energy. They are commonly used in various forms of commercial and industrial consumption. Producing oil is a complex task that requires special management and planning. This can result in a serious problem if the oil well is not operated properly. Oil engineers must have the necessary knowledge about the well’s status to perform their duties properly. This study pro-poses a linear regression method to predicate the oil production value. It takes into account various independent variables, such as the pressure, downhole temperature, and pressure tubing. The proposed method can accurately reach a very close prediction of the actual production value by achieving very interesting results at the end of this study.