Big Data Paradigm Research Articles

A Flexible Big Data System for Credibility-Based Filtering of Social Media Information According to Expertise

Nowadays, social networks have taken on an irreplaceable role as sources of information. Millions of people use them daily to find out about the issues of the moment. This success has meant that the amount of content present in social networks is unmanageable and, in many cases, fake or non-credible. Therefore, a correct pre-processing of the data is necessary if we want to obtain knowledge and value from these data sets. In this paper, we propose a new data pre-processing technique based on Big Data that seeks to solve two of the key concepts of the Big Data paradigm, data validity and credibility of the data and volume. The system is a Spark-based filter that allows us to flexibly select credible users related to a given topic under analysis, reducing the volume of data and keeping only valid data for the problem under study. The proposed system uses the power of word embeddings in conjunction with other text mining and natural language processing techniques. The system has been validated using three real-world use cases.

Quantifying the diversity of multiple time series with an ordinal symbolic approach.

The main motivation of this paper is to introduce the ordinal diversity, a symbolic tool able to quantify the degree of diversity of multiple time series. Analytical, numerical, and experimental analyses illustrate the utility of this measure to quantify how diverse, from an ordinal perspective, a set of many time series is. We have shown that ordinal diversity is able to characterize dynamical richness and dynamical transitions in stochastic processes and deterministic systems, including chaotic regimes. This ordinal tool also serves to identify optimal operating conditions in the machine learning approach of reservoir computing. These results allow us to envision potential applications for the handling and characterization of large amounts of data, paving the way for addressing some of the most pressing issues facing the current big data paradigm.

The Open Prison of the Big Data Revolution: False Consciousness, Faustian Bargains, and Digital Entrapment

Although some scholars raise alarm about societal harm emerging from Big Data practices, critical social theory (CST) Information Systems research on the structures and dynamics driving Big Data practices is rare. In this research commentary, we interrogate how tech firms use social practices and platform design to strategically manipulate individuals into accepting datafication and data assetization that accrue positive data network effects for themselves and mostly negative data network effects (economic loss, social and privacy harm) for individuals. We draw on the ideas of Heidegger and Marcuse to critically question the Big Data paradigm in order to develop better understanding of the social implications for individuals and society. Using the concepts of false consciousness, digital entrapment, and Faustian bargains, we critically inquire into the Big Data practices that keep us tethered to digital platforms. Specifically, we interrogate sociomaterial structures that socially condition individuals into a digital habitus and to identify themselves as homo digitalis, who view all their “relations” (social and economic) as digital. This social conditioning reproduces a false consciousness that constricts our worldview, undermines our rational choices, and enables the risky compromises we make with tech companies that manipulate and exploit us with their increasingly oppressive Big Data practices and related dark patterns. We critically analyze the case of Microsoft Viva to provide an illustration of how mundane digital tools can condition our reality and entrap us into an open prison. We argue that if we do not critically interrogate our false consciousness of the digital and understand how digital giants colonize our social systems by structurally embedding Big Data practices, we will continue to be susceptible to manipulation and digital entrapment. Ongoing risky compromises with tech firms will erode the very foundations of the “good life,” freedom, liberty, and personal privacy, and they will institutionalize the open prison. The CST explanation we propose and the research agenda we outline are meant to encourage research into solutions to the digital entrapment problem.History: Suprateek Sarker, Senior Editor; Robert Gregory, Associate Editor.Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2020.0588 .

Barriers and beliefs: a comparative case study of how university educators understand the datafication of higher education systems

In recent decades, higher education institutions around the world have come to depend on complex digital infrastructures. In addition to registration, financial, and other operations platforms, digital classroom tools with built-in learning analytics capacities underpin many course delivery options. Taken together, these intersecting digital systems collect vast amounts of data from students, staff, and faculty. Educators’ work environments—and knowledge about their work environments—have been shifted by this rise in pervasive datafication. In this paper, we overview the ways faculty in a variety of institutional status positions and geographic locales understand this shift and make sense of the datafied infrastructures of their institutions. We present findings from a comparative case study (CCS) of university educators in six countries, examining participants’ knowledge, practices, experiences, and perspectives in relation to datafication, while tracing patterns across contexts. We draw on individual, systemic, and historical axes of comparison to demonstrate that in spite of structural barriers to educator data literacy, professionals teaching in higher education do have strong and informed ethical and pedagogical perspectives on datafication that warrant greater attention. Our study suggests a distinction between the understandings educators have of data processes, or technical specifics of datafication on campuses, and their understanding of big picture data paradigms and ethical implications. Educators were found to be far more knowledgeable and comfortable in paradigm discussions than they were in process ones, partly due to structural barriers that limit their involvement at the process level.Graphical

Real-Time Streaming Analytics using Big Data Paradigm and Predictive Modelling based on Deep Learning

With the evolution of distributed streaming platforms analysing humongous time series data, which is streamed continuously from IoT devices become lot easier. In most of the IoT networks the data are in motion or in data centre/cloud. It is possible to process this data in real time similar to edge devices using the big data framework. In data intensive applications predictive analytics require more resources to perform complex computations. Apache Flink framework is capable of performing real time streaming of schema less data and scales very high in distributed environment with low latency, it is used to collect and store the data in the cloud. This work suggests a suitable environment to collect, transport, preprocess and aggregate the data stream to perform predictive analytics using deep learning models. Deep learning automatically extracts features and builds models after training, it has the potential to solve problems that can't be solved by conventional machine learning models. Therefore, the use of algorithms based on deep learning is recommended for forecasting temporal data. Also, we discuss a number of different deep learning forecasting models and analyse the performance of different deep learning forecasting models in order to determine which one is the effective model for single step, multi step and multi variant methods based on error functions with respect to streamed sensor data.

Bibliometric analysis of electroencephalogram research in mild cognitive impairment from 2005 to 2022.

Electroencephalogram (EEG), one of the most commonly used non-invasive neurophysiological examination techniques, advanced rapidly between 2005 and 2022, particularly when it was used for the diagnosis and prognosis of mild cognitive impairment (MCI). This study used a bibliometric approach to synthesize the knowledge structure and cutting-edge hotspots of EEG application in the MCI. Related publications in the Web of Science Core Collection (WosCC) were retrieved from inception to 30 September 2022. CiteSpace, VOSviewer, and HistCite software were employed to perform bibliographic and visualization analyses. Between 2005 and 2022, 2,905 studies related to the application of EEG in MCI were investigated. The United States had the highest number of publications and was at the top of the list of international collaborations. In terms of total number of articles, IRCCS San Raffaele Pisana ranked first among institutions. The Clinical Neurophysiology published the greatest number of articles. The author with the highest citations was Babiloni C. In descending order of frequency, keywords with the highest frequency were "EEG," "mild cognitive impairment," and "Alzheimer's disease". The application of EEG in MCI was investigated using bibliographic analysis. The research emphasis has shifted from examining local brain lesions with EEG to neural network mechanisms. The paradigm of big data and intelligent analysis is becoming more relevant in EEG analytical methods. The use of EEG to link MCI to other related neurological disorders, and to evaluate new targets for diagnosis and treatment, has become a new research trend. The above-mentioned findings have implications in the future research on the application of EEG in MCI.

Fleet management systems in Logistics 4.0 era: a real time distributed and scalable architectural proposal

In an era marked by the big data paradigm and ubiquitous computing systems, it is increasingly common to see devices embedded in any type of object with the aim of collecting and sharing owned or read data from its environment. This type of interconnection between devices is known as the Internet of Things (IoT). In particular in the logistics sector, vehicles are equipped with control units that are capable of monitoring a large number of parameters to ensure the correct operation of the vehicle. In addition, they are now able to share this data in near real time so that this information can be accessed and analysed at any time. However, due to the large amount of shared data, the frequency of data generation and delivery, and the high potential for growth in the number of devices, traditional technologies and architectures are not able to meet the performance demands of these real time decision making systems. In this article we describe and evaluate the benefits and potential trade-offs of implementing services based on a distributed and scalable architecture, called RAI4.0, in a truck fleet management company, which currently has 20,000 on-board devices and expects to grow to 80,000 devices in the next 2 years. With the change of architecture, the company expects to be able to implement near real-time services to monitor and notify its drivers of driving tips and diagnose possible vehicle failures in advance, among others.

BIG DATA AND ARTIFICIAL INTELLIGENCE

The Big Data embodies a technology that permits the storage, processing, and management of broad and complex data sets in which traditional data processing applications are not applicable. These sets of data are usually characterized by a substantial volume of information that they carry, variety, versatility in terms of the format in which they are written, as well as a high-speed ingress which is often greater than the speed of processing. A particular challenge is the data which are coming from the Internet of Things (IoT) “world” that is constantly expanding and which already consists of several billion devices that can measure various parameters in the environment, communicate, process and transmit information. The data streams emanating from these devices are changing traditional approaches to data management and contribute to the emergence of the Big Data paradigm. This paper discusses the characteristics of the IoT infrastructure in terms of vast scale sensor applications and a possibility of connectivity of sensor networks, as well as various techniques of collection, storage, archiving and processing of the data on the cloud.

Data mining tools for complex socio-economic processes and systems

The paper considers discovering new and potentially useful information from large amounts of data that actualizes the role of developing data mining tools for complex socio-economic processes and systems based on the principles of the digital economy and their processing using network applications. The stages of data mining for complex socio-economic processes and systems were outlined. The algorithm of data mining was considered. It is determined that the previously used stages of data mining, which were limited to the model-building process, can be extended through the use of more powerful computer technology and the emergence of free access to large amounts of multidimensional data. The available stages of data mining for complex socio-economic processes and systems include the processes of facilitating data preparation, evaluation, and visualization of models, as well as in-depth learning. The data mining tools for complex socio-economic processes and systems in the context of technological progress and following the big data paradigm were identified. The data processing cycle has been investigated; this process consists of a series of steps starting with the input of raw data and ending with the output of useful information. The knowledge obtained at the data processing stage is the basis for creating models of complex socio-economic processes and systems. Two types of models (descriptive and predictive) that could be created in the data mining process were outlined. Algorithms for estimating and analyzing data for modeling complex socio-economic processes and systems in accordance with the pre-set task were determined. The efficiency of introducing neural networks and deep learning methods used in data mining was analyzed. It was determined that they would allow effective analysis and use of the existing large data sets for operational human resources management and strategic planning of complex socio-economic processes and systems.

Big Data Research in India: A Scientometric Review

Big Data paradigm has grown rapidly in recent years, resulting in significant research from a tremendous multitude of perspectives. A literature review is an important methodology for gathering information about the ongoing activity of science. There are several methods including bibliographic or scientometric studies in reviewing the research literature. This paper studied with a scientometric analysis of research work on the emerging field ‘Big Data’ in India using a Python software called ‘ScientoPy’. A data set of 3080 papers on Big Data Research in India was collected which are indexed in two major scientific databases, i.e. Clarivate Web of Science and Scopus over a 10-year period (2012–2021). This aims to examine the main authors, country affiliation, most published authors, top research outputs, and emerging themes to provide insight into recent developments in the field.

Theoretical Development of Connectivism through Innovative Application in China

As a learning theory that reveals a new learning in the Internet environment, connectivism has become a popular academic topic at the forefront of online learning. The MOOC Research Team at the Distance Education Research Centre at Beijing Normal University designed and developed the first massive open online course, adapting a connectivist (cMOOC) approach in China. Using the data collected from six offerings of the cMOOC over 3 years, the big data paradigm was used for data analysis including complex network analysis, content analysis, text mining, behaviour sequence analysis, epistemic network analysis, and statistical and econometric models. This paper summarizes the findings of the patterns of connectivist learning, including a) the basic characteristics and evolutional patterns of complex networks, b) the characteristics and modes of knowledge production, c) the patterns of instructional interactions, and d) the relationships between pipe and content and between facilitators and learners. It is expected that the outcome of this study could make contributions to understanding the changes of online learning in depth and further promote the theoretical development and practical application of a connectivist approach.

Modern Urban Forestry for Modern Cities: Technology Challenges and Opportunities for the Remote Sensing of Urban Forests

Background: As human populations urbanize, urban forests in many areas are decreasing in canopy extent due to disruptions on several fronts, including novel pests and diseases, climate change, and changing land uses. Methods: A review of the remote sensing, computing, and environmental literature was performed to provide an overview of current technology capabilities and to detail an agenda for a modern approach to urban forestry challenges. How to prepare current and future professionals to collect and analyze “Big Data,” how to implement results, and what communication skills are needed in a modern world to provide resilient urban forests in the connected future were also reviewed. Results: This paper outlines an agenda for how the urban forestry professions can identify, analyze, and manage emergent disruptions to continue to provide urban forest benefits to residents in its shade. Current remote-sensing systems, the paradigm of Big Data, and collection and analysis platforms are discussed, and relevant scenarios are provided to guide insight into managing forests with a rejuvenated perspective using remote-sensing hardware and software. Conclusions: Modern cities will require modern digital urban forestry management, and current and future professionals must be able to access and utilize technology, sensors, and Big Data to effectively perform vegetation management and communication tasks. This paper details the framework for a new era of modern urban forest management in highly connected, resilient cities.

From indirect to direct contacts on Facebook: A big-data approach to the making of triadic network closure.

The rise of the Big Data paradigm has made it more feasible to track how personal networks evolve on social media, where auto-generated contact records and fine-grained temporal data sequences help capture how and when interpersonal ties and contacts change their roles. Using a sample of matched survey data and social media records, we investigated the mechanisms by which indirect contacts ("degree-2 alters") transform into direct contacts ("degree-1 alters") from a Facebook user's (ego's) point of view. To highlight the temporal sequences, we assigned different roles to the same alters depending on how each of them is connected with ego at different periods of time. Multilevel event history analyses pinpoint several online actions and network features of ego, degree-1 alters, and degree-2 alters, as the key factors that contribute to the transformation from indirect contacts into direct contacts.

Exploring New Vista of Intelligent Recommendation Framework for Tourism Industries: An Itinerary through Big Data Paradigm

Big Data is changing how organizations conduct operations. Data are assembled from multiple points of view through online quests, investigation of purchaser purchasing conduct, and then some, and industries utilize it to improve their net revenue and give an overall better experience to clients. Each of these organizations must figure out how to improve the general client experience and meet every client’s novel necessities, and big data helps with this cycle. Through the utilization and reviews of Big Data, travel industry organizations can study the inclinations of more modest portions of their intended interest group or even about people in some cases. In this paper, a Crow Search Optimization-based Hybrid Recommendation Model is proposed to get accurate suggestions based on clients’ preferences. The hybrid recommendation is performed by combining collaborative filtering and content-based filtering. As a result, the advantages of collaborative filtering and content-based filtering are utilized. Moreover, the intelligent behavior of Crows’ assists the proper selection of neighbors, rating prediction, and in-depth analysis of the contents. Accordingly, an optimized recommendation is always provided to the target users. Finally, performance of the proposed model is tested using the TripAdvisor dataset. The experimental results reveal that the model provides 58%, 58.5%, 27%, 24.5%, and 25.5% better Mean Absolute Error, Root Mean Square Error, Precision, Recall, and F-Measure, respectively, compared to similar algorithms.

EARTH OBSERVATION AND GLOBAL NAVIGATION SATELLITE SYSTEMS ANALITICAL REPORT PART II (TIMING & SYNCHRONISATION OF TELECOMMUNICATION NETWORKS, MARITIME AND INLAND WATERWAYS, RAIL AND AUTOMOTIVE TRANSPORT)

The EU Space Programme is a business growth enabler that stimulates the economy and pushes the bar of innovation. The EUSPA EO & GNSS Market Report is the ultimate guide to anyone who seeks to make the EU Satellite Navigation and Earth Observation technologies part of their business plan and develop new space downstream applications. More than ever society relies on innovative solutions to deal with the big data paradigm. Earth Observation (EO) and Global Navigation Satellite System (GNSS) data is becoming increasingly important to these innovative solutions through dozens of applications that are emerging or already in use by citizens, businesses, governments, industry, international organisations, NGOs and researchers around the world. The study provides analytical information on the dynamic GNSS and EO markets, along with indepthanalyses of the latest global trends and developments through illustrated examples and use cases. Using advanced econometric models, it also offers market evolution forecasts of GNSS shipments or EO revenues spanning to 2031. This article represent the brief overview essential role of space data across market segments including, timing & synchronisation of telecommunication networks, maritime and inland waterways, rail and automotive transport.

EARTH OBSERVATION AND GLOBAL NAVIGATION SATELLITE SYSTEMS ANALITICAL REPORT PART I (AVIATION AND SPACE)

The EU Space Programme is a business growth enabler that stimulates the economy and pushes the bar of innovation. The EUSPA EO & GNSS Market Report is the ultimate guide to anyone who seeks to make the EU Satellite Navigation and Earth Observation technologies part of their business plan and develop new space downstream applications. More than ever society relies on innovative solutions to deal with the big data paradigm. Earth Observation (EO) and Global Navigation Satellite System (GNSS) data is becoming increasingly important to these innovative solutions through dozens of applications that are emerging or already in use by citizens, businesses, governments, industry, international organisations, NGOs and researchers around the world. The study provides analytical information on the dynamic GNSS and EO markets, along with indepth analyses of the latest global trends and developments through illustrated examples and use cases. Using advanced econometric models, it also offers market evolution forecasts of GNSS shipments or EO revenues spanning to 2031. With a focus on Galileo/EGNOS and Copernicus, the report highlights the essential role of space data across 17 market segments including, Agriculture; Aviation and Drones; Biodiversity, Ecosystems and Natural Capital; Climate Services; Consumer Solutions, Tourism, and Health; Emergency Management and Humanitarian Aid; Energy and Raw Materials; Environmental Monitoring; Fisheries and Aquaculture; Forestry; Infrastructure; Insurance and Finance; Maritime and Inland Waterways; Rail; Road and Automotive; Urban Development and Cultural Heritage; and Space. This article represent the brief overview essential role of space data across 2 market segments including, Aviation and Drones and Rail, Road and Automotive.

Adaptation of the Lean 6S Methodology in an Industrial Environment under Sustainability and Industry 4.0 Criteria

Industry 4.0 paradigms have a positive influence on standard operating procedures, methodologies used in Lean Manufacturing techniques and management models with sustainability criteria. Interdependencies and correlations have been found between Lean systems and Industry 4.0. The Lean principles of avoiding waste and zero defects are related to the cloud and big data paradigms. In a current workplace, there has been an exponential increase in digital information and the need to generate direct commitments to environmental management. This situation forces us to innovate and improve the management methodologies and models used in the industrial environment. The Lean 6S methodology must adapt and respond to new demands. In this work, an update of the Lean 6S methodology is carried out to guarantee increased productivity in the workplace through the organization of industrial resources, both physical and digital. A revision of the implementation procedure is proposed, which includes activities that generate a direct commitment to sustainability and the organization of digital information, through a proposal for an organizational architecture of Industry 4.0 technologies.

GENERAL ASSESSMENT OF THE PROSPECTS FOR THE APPLICATION OF THE BIG DATA PARADIGM IN SOCIO-ECONOMIC SYSTEMS

The paper assesses the prospects for the application of the big data paradigm in socio-economic systems through the analysis of factors that distinguish it from the well-known scientific ideas of data synthesis and decomposition. The idea of extracting knowledge directly from big data is analyzed. The article compares approaches to extracting knowledge from big data: algebraic and multidimensional data analysis used in OLAP-systems (OnLine Analytical Processing). An intermediate conclusion is made about the advisability of dividing systems for working with big data into two main classes: automatic and non-automatic. To assess the result of extracting knowledge from big data, it is proposed to use well-known scientific criteria: reliability and efficiency. It is proposed to consider two components of reliability: methodical and instrumental. The main goals of knowledge extraction in socio-economic systems are highlighted: forecasting and support for making management decisions. The factors that distinguish big data are analyzed: volume, variety, velocity, as applied to the study of socio-economic systems. The expediency of introducing a universe into systems for processing big data, which provides a description of the variety of big data and source protocols, is analyzed. The impact of the properties of sample populations from big data: incompleteness, heterogeneity, and non-representativeness, the choice of mathematical methods for processing big data is analyzed. The conclusion is made about the need for a systemic, comprehensive, cautious approach to the development of fundamental decisions of a socio-economic nature when using the big data paradigm in the study of individual socio-economic subsystems.

Regulating Biobanks: An ethical analysis of the Spanish law and the new challenges of the big data-driven biomedical research

In the European landscape, Spain represents a positive reference point when it comes to biobank regulation. Indeed, at the beginning of XXI century, the Spanish legislation has promptly responded to challenges posed by new biotechnologies and advances in genomics in the field of biomedical research by enacting in 2007 the Ley de Investigación Biomédica in order to keep up with the paradigm shift. Over the past 10 years, this Spanish framework along with the Real Decreto 1716/2011 has hold the merit to tackle the most controversial ethical issues related to use of human samples and personal data in biomedical research and biobanking (e.g. broad consent, secondary uses, governance, etc.). However, today the regulation of biomedical research and biobanks has to deal with big data, artificial intelligence and data-intensive research which have brought a number of challenges and controversies. The aim of this paper is two-fold. First, I will analyse from an ethical point of view the merits of Spanish regulation on biobanking in order to draw some lessons for the still unregulated situation in other Member States. Secondly, I will discuss the big data paradigm shift in biomedical research and question if the ethical and legal framework introduced the Spanish law at the beginning of the century is still able to hold the ground with the new contextual and societal challenges. In this respect, I will identify some opportunities for implementation and suggest strategies to achieve them in the specific context of biobanks.

Big data does not equal big knowledge. A need for causal insights

BACKGROUND AND AIM: There is hope that the bigger the data, the bigger the knowledge will become. Very large data are or will be made available and shall bring robust and replicable effect estimates in exposome research. While larger data and complementary sources of data will, with little doubt, bring robust effect sizes to estimate how environmental, toxic, social, and biological exposures are associated with health outcomes. We may also expect that larger (and precise) data will allow us to go beyond common effects (as an analogy to common genetic variants) and detect rare or small effects. Moving on into the paradigm of big data, it may be wise to remind each other that correlation is not proof of causality. METHODS: Although bigger (observational) data will bring confidence on the strength of some association, relationships in large data are similarly affected by important selection and structural biases, confounding effects, and measurement errors. One strength of the European Human Exposome Network is to bring multiple study designs and dynamic longitudinal analytical strategies to help to triangulate the evidence. RESULTS:We shall, for example, use negative control, quasi-experimental designs, interventions, and instrumental variables (e.g. Mendelian randomization or instrument based on exogenous factors, i.e. environmental policies) to infer causation and identify causative pathways. CONCLUSIONS:These causal designs and approaches however are not always applicable or available. One should also seek for additional ways to gain confidence on the plausibility of causal insights through replication, triangulation, and integration of prior knowledge which is made available in the European Human Exposome Network. KEYWORDS: European Human Exposome Network, Big data, big knowledge, causal insights