Big Data Methodologies Research Articles

Enabling data-driven process dynamic modeling for extractive leaching and chemical precipitation

To address the limitations of static models and gain insight into the processes of extractive leaching and chemical precipitation, a data-driven dynamic modeling strategy is proposed using a Lithium-ion battery recycling case study. The data correlations among pH, temperature, redox potential, conductivity and system state are investigated. Predictive models are then developed to describe the system state online and are employed as surrogate models for time-intensive offline chemical analyses. This enables further process optimization, such as time-saving measures and improved process efficiency through dynamic parameter studies. The proposed strategy serves as a guideline for dynamic modeling and integrates big data methodologies into chemical engineering.

Additive Hazards Regression Analysis of Massive Interval-Censored Data via Data Splitting

With the rapid development of data acquisition and storage space, massive data sets exhibited with large sample size emerge increasingly and make more advanced statistical tools urgently need. To accommodate such big volume in the analysis, a variety of methods have been proposed in the circumstances of complete or right censored survival data. However, existing development of big data methodology has not attended to interval-censored outcomes, which are ubiquitous in cross-sectional or periodical follow-up studies. In this work, we propose an easily implemented divide-and-combine approach for analyzing massive interval-censored survival data under the additive hazards model. We establish the asymptotic properties of the proposed estimator, including the consistency and asymptotic normality. In addition, the divide-and-combine estimator is shown to be asymptotically equivalent to the full-data-based estimator obtained from analyzing all data together. Simulation studies suggest that, relative to the full-data-based approach, the proposed divide-and-combine approach has desirable advantage in terms of computation time, making it more applicable to large-scale data analysis. An application to a set of interval-censored data also demonstrates the practical utility of the proposed method.

Using Data Mining Principles in Implementing Predictive Analytics to Different Areas

This study delves into the realm of information-based knowledge discovery technologies and underscores the growing necessity for extensive data representation to enhance the management of care and mitigate the financial costs associated with promoting long-term care. The proliferation of information collected and disseminated through the Internet has reached unprecedented levels in the context of long-term financial health statistics, posing a challenge for businesses to effectively leverage this wealth of data for research purposes. The explicit specification of costs becomes paramount when dealing with substantial volumes of data. Consequently, the literature on the application of big data in logistics is categorized based on the nature of methods employed, such as explanatory, predictive, regulatory, strategic, and operational approaches. This includes a comprehensive examination of how big data analysis is applied within large corporations. In the healthcare domain, the study contributes to the evaluation of usability by providing a framework to analyze the maturity of structures at four distinct levels. The emphasis is particularly on the pivotal role played by predictive analytics in the healthcare industry through big data methodologies. Furthermore, the study advocates for a paradigm shift in management's perception of large business data sets, urging them to view these as strategic resources that must be seamlessly integrated into the company. This integration is seen as imperative for achieving comprehensive business analysis and staying competitive in the ever-evolving landscape of healthcare. The study concludes by shedding light on the limitations inherent in the research and delineating the specific focus areas that have been addressed.

The Main Controlling Factors of the Cambrian Ultra-Deep Dolomite Reservoir in the Tarim Basin

The genesis of deep-to-ultra-deep dolomite reservoirs in the Tarim Basin is crucial for exploration and development. The Cambrian subsalt dolomite reservoirs in the Tarim Basin are widely distributed, marking significant prospects for ultra-deep reservoir exploration. Based on big data methodologies, this study collects and analyzes porosity and permeability data of carbonate reservoirs in the western Tarim Basin, specifically targeting the Cambrian deep-oil and gas-reservoir research. Through an examination of the sedimentary evolution and distribution of carbonate–evaporite sequences, and considering sedimentary facies, stratigraphic sediment thickness, fault zone distribution, and source-reservoir assemblages as primary reference factors, the study explores the macro-distribution patterns of porosity and permeability, categorizing three favorable reservoir zones. The controlling factors for the development of Cambrian carbonate reservoirs on the western part of the Tarim Basin are analyzed from the perspectives of sedimentary and diagenetic periods. Factors such as tectonic activity, depositional environment, microbial activity, and pressure dissolution are analyzed to understand the main causes of differences in porosity and permeability distribution. Comprehensive analysis reveals that the porosity and permeability of the Series2 carbonate reservoirs are notably high, with extensive distribution areas, particularly in the Bachu–Tazhong and Keping regions. The geological pattern of “Three Paleo-uplifts and Two Depressions” facilitated the formation of inner-ramp and intra-platform shoals, creating conducive conditions for the emergence of high-porosity reservoirs. The characteristics of reservoir development are predominantly influenced by diagenetic and tectonic activities. The Miaolingian is chiefly affected by diagenesis, featuring high permeability but lower porosity and smaller distribution range; dolomitization, dissolution, and filling processes under a dry and hot paleoclimate significantly contribute to the formation and preservation of reservoir spaces. In the Furongian, the Keping and Bachu areas display elevated porosity and permeability levels, along with substantial sedimentary thickness. The conservation and development of porosity within thick dolomite sequences are mainly governed by high-energy-particulate shallow-shoal sedimentary facies and various dissolution actions during diagenesis, potentially indicating larger reserves.

Smart Data Driven Decision Trees Ensemble Methodology for Imbalanced Big Data

Differences in data size per class, also known as imbalanced data distribution, have become a common problem affecting data quality. Big Data scenarios pose a new challenge to traditional imbalanced classification algorithms, since they are not prepared to work with such amount of data. Split data strategies and lack of data in the minority class due to the use of MapReduce paradigm have posed new challenges for tackling the imbalance between classes in Big Data scenarios. Ensembles have been shown to be able to successfully address imbalanced data problems. Smart Data refers to data of enough quality to achieve high-performance models. The combination of ensembles and Smart Data, achieved through Big Data preprocessing, should be a great synergy. In this paper, we propose a novel Smart Data driven Decision Trees Ensemble methodology for addressing the imbalanced classification problem in Big Data domains, namely SD_DeTE methodology. This methodology is based on the learning of different decision trees using distributed quality data for the ensemble process. This quality data is achieved by fusing random discretization, principal components analysis, and clustering-based random oversampling for obtaining different Smart Data versions of the original data. Experiments carried out in 21 binary adapted datasets have shown that our methodology outperforms random forest.

Get the hammer out!

This paper focuses on revealing how the interplay between algorithmic interactions and the intuitive ways humans navigate digital environments can be researched through a multi-method approach to collecting and critically examining data from online platforms. We use a case study that looks at the role that social media engagement by transnational activists, local activists and celebrities played in amplifying an offline protest by group of women in India. Grounded in a critical feminist perspective, this paper uses multiple methods to demonstrate how the amplification of local protesters work through an interplay of human action and platform algorithmics. We conduct an algorithmic ethnography involving the examination of computational systems shaping online interactions. We examine the digital emergence and recognition of the women of Shaheen Bagh as subaltern political agents/subjects. Understanding of the interplay between online and offline visibility and strategic planning is highlighted. We conduct close readings of small data clusters that emerge within big data networks. We challenge the overreliance on big data methodologies and the fetishization of in-person ethnography (Bishop 2018) over digital ethnography.

Aging in the Digital Age: Public Beliefs About the Potential of Virtual Reality (VR) for the Aging Population

This paper responds to the scholarly call for investigating the role of technology consumption in promoting healthy aging—it aims to identify the public's beliefs about the potential of virtual reality technology (VR) for the well-being of older adults. The study utilized a big-data methodology and employed machine learning to scrape comments made by social media users on the most popular YouTube videos that discuss older people's use of VR technology. After the data cleaning process, the study was left with 1,917 comments out of 3,952 that were then subject to analysis through thematic, emotion, and sentiment analyses. The findings revealed three themes of the public beliefs: 1) empowerment through technology, generational adaptability, and learning; 2) social and ethical implications of VR for healthy aging; and 3) cognitive and physical engagement. The emotion and sentiment analyses indicated that the general public has a predominantly positive perception of the application of virtual reality technology for older people. In summary, the synthesis of the results from the three analyses suggested that VR has the potential to empower seniors and promote mental and physical activity while also highlighting the importance of maintaining genuine connections and ethical standards.

Literature Review on the Smart City Resources Analysis with Big Data Methodologies

This article provides a systematic literature review on applying different algorithms to municipal data processing, aiming to understand how the data were collected, stored, pre-processed, and analyzed, to compare various methods, and to select feasible solutions for further research. Several algorithms and data types are considered, finding that clustering, classification, correlation, anomaly detection, and prediction algorithms are frequently used. As expected, the data is of several types, ranging from sensor data to images. It is a considerable challenge, although several algorithms work very well, such as Long Short-Term Memory (LSTM) for timeseries prediction and classification.

Meningkatkan Kualitas Official Statistics Melalui Integrasi Big Data

This paper delves into the transformative impact of incorporating big data into official statistical practices. The objective is to elevate the overall quality of data by leveraging the potential of big data analytics. The study employs a comprehensive methodology that encompasses data collection, processing, and analysis. The primary aim is to explore how the integration of big data can enhance the accuracy, timeliness, and relevance of official statistics. The research methodology involves the systematic collection of large-scale datasets, followed by advanced analytics techniques to extract meaningful insights. The findings underscore the substantial improvements achieved in the precision and reliability of official statistics through the infusion of big data methodologies. The transformative potential of integrating big data into official statistics is explored, offering valuable insights for policymakers, researchers, and practitioners in the field.

Моделирование системы мониторинга аграрных экологических систем на основе больших данных

Abstract. Due to population growth and food demand, the monitoring of agrarian ecological systems is becoming increasingly important. This is due to the expected use of resources, increased yields and the impacts of agricultural systems in the face of climate change and increasing anthropogenic pressure. The use of such technologies makes it possible to obtain more accurate and objective data on the state of agricultural ecosystems, which, in turn, is based on decisions made aimed at improving the management of agricultural ecosystems and optimizing agricultural practices. Purpose. In this scientific paper, the purpose is to present the results of the assessment of agricultural ecological systems, developed on the basis of the use of Big Data. Methods. The authors of the article analyze the methods of monitoring agroecosystems and justify a new observation that will improve the quality and control of monitoring results. The main emphasis is placed on the use of big data analysis and machine learning methods to obtain more accurate and objective information about the state of agricultural ecosystems. Scientific novelty. The authors have carried out modeling of monitoring systems for agrarian ecological systems based on big data methodology. This represents a transition from classical approaches to more efficient and accurate ones, which is a significant step forward in this field of research. Results. The new model for monitoring agrarian ecological systems provides opportunities for a more accurate and objective study and assessment of the state of agroecosystems. It also allows you to make informed decisions based on the information received, which is an important guarantee for the sustainable development of the agricultural sector. In conclusion, the authors consider the possibilities for improving efficiency and its application models in various areas of agricultural activity.

Assessing the risk of using hazardous drugs in Hospital-at-Home Units: a big data study protocol

ObjectiveThis article describes a study protocol for the implementation of quality and traceability control in the hazardous medication circuit through an analysis of risks and the development and introduction of a Big Data-based software application aimed at performing a continuous and dynamic audit of the whole system. MethodA standardized graphical modeling tool called Business Process Model Notation will be used to generate a detailed description of each of the stages in the hazardous medication circuit with a view to ensuring full traceability of the system. The information on each stage will be collected in a flowchart, which will be used —together with each event's likelihood of occurrence and severity— as a basis to calculate the criticality index of the different control points established and to determine any control measures that may be required. The flowcharts will also be used to develop the technological support needed to capture all such data as may be relevant to the model. Proper quality control of the process will be ensured by client software agents intended to allow automatic application of efficient data processing tools at the different phases. In addition, Big Data methodologies, in particular machine learning, will be used to develop algorithms based on the repository of generated data to come up with patterns capable of improving the protocols to be applied. Lastly, proper operation of the process will be ensured by means of clinical-pharmaceutical verification and a full technical-documentary review of control and registration systems. ConclusionsThe development of a risk management system based on mobile technology will allow integration of hazardous drugs into a standardized system, ensuring the safety, quality, and traceability of the hazardous medication handling process.

토픽모델링을 이용한 ESG 연구현황 분석

[Purpose]The purpose of this study is to analyze the research trends of ESG research in Korea, focusing on the fields of business administration and accounting, using topic modeling.
 [Methodology]This study focuses on the keyword ‘ESG’ published in domestic academic journals. To this end, as of June 30, 2023, a total of 404 studies of business administration and accounting studies calculated with ‘ESG’ as a search term in the Korea Citation Index are the subject of research. Specifically, morpheme analysis was performed using R, keyword frequency analysis was performed, and frequency analysis and Big-data analysis(data mining, topic modeling) were performed through the extracted keywords.
 [Findings]As a result of this study, five topics were calculated. Specifically, Topic 1 is the role of the environment for management sustainability, Topic 2 is Implications for enhancing corporate value, Topic 3 is Strategy according to economic conditions, Topic 4 is Decision-making at the governance level, Topic 5 is Evaluation and utilization It is a responsibility.
 [Implications]This study is different in that it analyzes research trends using the big data analysis method with the keyword ‘ESG’ limited to business administration and accounting among social sciences. In the meantime, many studies have been conducted in business administration and accounting, but there are not many studies that apply big data methods such as topic modeling. This study has significance in that it goes one step further from the existing research methodology, reduces the subjective opinions of researchers, and identifies macroscopic research trends based on a lot of data through the Big-data methodology.

Prevalence of cancer in patients with hypothyroidism: Analysis using big data tools

ObjectiveTo evaluate the frequency of different types of cancer in patients diagnosed with hypothyroidism using big data methodology on the Savana Manager platform. MethodsAn observational, retrospective study was carried out using electronic medical record (EMR) data from the Hospital Universitario Puerta de Hierro Majadahonda (Madrid). Information from the EMRs was extracted using artificial intelligence techniques and analysed using the Savana Manager v3.0 software. Searches were performed using the term “hypothyroidism” and the terms corresponding to the tumours analysed. ResultsOf a total population of 506,749 patients, 23,570 (4.7%) were diagnosed with hypothyroidism. Patients with this diagnosis had a significantly higher frequency of cancer than that found in non-hypothyroid subjects (OR 2.09, 95% confidence interval [CI] 2.01−2.17). This higher frequency was found both in women (OR 1.99, 95% CI 1.90−2.08) and in men (OR 2.83, 95% CI 2.63−3.05). However, this higher frequency of cancer was not observed in hypothyroid patients older than 60 years (OR 0.97, 95% CI 0.92−1.02). Although the frequency of most of the neoplasms studied individually was higher in the population with hypothyroidism, we observed that hypothyroid patients over 60 years of age had a significant decrease in the frequency of prostate, lung, colorectal, and liver cancer. ConclusionData from this hospital cohort suggest that there is a significant association between the diagnosis of hypothyroidism and cancer. However, this association is less evident in hypothyroid patients older than 60 years.

The social perceptions of young children’s use of smart devices in South Korea: Evidence from big data methodologies

The study explored the social perceptions of young children’s use of smart devices in South Korea using big data methodologies. Big data methodologies allowed to uncover underlying thoughts and feelings about young children’s use of smart devices that had not been discovered in existing studies. The study extracted raw data from three different groups: the public, the journalist, and academia. Then, the study conducted keyword frequency, sentiment analysis, and CONCOR analysis with UCINET 6.0. The results of the study revealed that each group viewed young children’s use of smart devices in a different way. The public was interested in effective use of smart devices while the journalist focused on educational aspects. The academia focused on parents’ perception of smart devices from a developmental perspective. Regarding the results of sentiment analysis, they showed that each group had different opinion on young children’s use of smart devices. The public had an ambivalent attitude toward young children’s use of smart devices. While the journalist showed a positively inclined attitude, the academic had a negatively inclined attitude.

Linguistic Methodologies to Surveil the Leading Causes of Mortality: Scoping Review of Twitter for Public Health Data.

Twitter has become a dominant source of public health data and a widely used method to investigate and understand public health-related issues internationally. By leveraging big data methodologies to mine Twitter for health-related data at the individual and community levels, scientists can use the data as a rapid and less expensive source for both epidemiological surveillance and studies on human behavior. However, limited reviews have focused on novel applications of language analyses that examine human health and behavior and the surveillance of several emerging diseases, chronic conditions, and risky behaviors. The primary focus of this scoping review was to provide a comprehensive overview of relevant studies that have used Twitter as a data source in public health research to analyze users' tweets to identify and understand physical and mental health conditions and remotely monitor the leading causes of mortality related to emerging disease epidemics, chronic diseases, and risk behaviors. A literature search strategy following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) extended guidelines for scoping reviews was used to search specific keywords on Twitter and public health on 5 databases: Web of Science, PubMed, CINAHL, PsycINFO, and Google Scholar. We reviewed the literature comprising peer-reviewed empirical research articles that included original research published in English-language journals between 2008 and 2021. Key information on Twitter data being leveraged for analyzing user language to study physical and mental health and public health surveillance was extracted. A total of 38 articles that focused primarily on Twitter as a data source met the inclusion criteria for review. In total, two themes emerged from the literature: (1) language analysis to identify health threats and physical and mental health understandings about people and societies and (2) public health surveillance related to leading causes of mortality, primarily representing 3 categories (ie, respiratory infections, cardiovascular disease, and COVID-19). The findings suggest that Twitter language data can be mined to detect mental health conditions, disease surveillance, and death rates; identify heart-related content; show how health-related information is shared and discussed; and provide access to users' opinions and feelings. Twitter analysis shows promise in the field of public health communication and surveillance. It may be essential to use Twitter to supplement more conventional public health surveillance approaches. Twitter can potentially fortify researchers' ability to collect data in a timely way and improve the early identification of potential health threats. Twitter can also help identify subtle signals in language for understanding physical and mental health conditions.

Scalable approach for high-resolution land cover: a case study in the Mediterranean Basin

The production of land cover maps is an everyday use of image classification applications on remote sensing. However, managing Earth observation satellite data for a large region of interest is challenging in the task of creating land cover maps. Since satellite imagery is getting more precise and extensive, Big Data techniques are becoming essential to handle the rising quantity of data. Furthermore, given the complexity of managing and analysing the data, defining a methodology that reduces the complexity of the process into different smaller steps is vital to data processing. This paper presents a Big Data methodology for creating land cover maps employing artificial intelligence algorithms. Machine Learning algorithms are contemplated for remote sensing and geodata classification, supported by explainable artificial intelligence. Furthermore, the process considers aspects related to downloading data from different satellites, Copernicus and ASTER, executing the pre-processing and processing of the data in a distributed environment, and depicting the visualisation of the result. The methodology is validated in a test case for er map of the Mediterranean Basin.

Artificial intelligence and education governance

Artificial intelligence (AI) and big data methodologies have provided a vast possibility of applications in different public policy sectors. AI and big data provide essential innovations in school teaching and learning practices, curriculum, and management in education, transforming the entire formulation and implementation of education policy. The possibilities of AI in education present a high potential for improvement but a series of new risks and threats that question the benefits of adopting AI in education. This article discusses the possibilities and limits of artificial intelligence in education, observing the institutionalization of new governance practices in education that emerge with digital technologies. To discuss these new governance practices, we deal with the Beijing Consensus on artificial intelligence in education and the challenges of building new education governance. In addition, we discuss potential impacts of AI on education, with a special focus on social justice.

An Energy-Based Big Data Framework to Estimate the Young’s Moduli of the Soils Drilled during the Execution of Continuous Flight Auger Piles

Understanding the soil mass and how it behaves is determinant to the quality and reliability of a foundation design. Normally, such behavior is predicted based on laboratory and in situ tests. In the big data era, instead of executing more tests, engineers should understand how to take advantage of ordinary execution procedures to obtain the parameters of interest. Sensors are key components in engineering big data frameworks, as they provide a large number of valuable measured data. In particular, the building process (excavation and concreting) of continuous flight auger piles (CFAPs) can be fully monitored by collecting data from sensors in the drilling machine. This makes this type of pile an ideal candidate to utilize a big data methodology to indirectly obtain some constitutive parameters of the soil being drilled. Thus, in the present paper, the drilling process of CFAPs is modeled by a new physical model which predicts the energy spending during the execution of this type of pile. This new model relies on other fundamental properties of the soils drilled, such as unit weight, cohesion and internal friction angle. In order to show the applicability of the big data methodological framework hereby developed, a case study was conducted. A work site in Brasília-DF, Brazil, was studied and the execution of three CFAPs was monitored. Soil surveys were carried out to identify the soil strata in the site and, therefore, to validate the estimates of Young’s moduli provided by the new formulas. The 95% confidence intervals of Young’s moduli obtained for silty clay, clayey silt and silt were, in MPa, [14.56, 19.11], [12.26, 16.88] and [19.65, 26.11], respectively. These intervals are consistent with literature reports for the following materials: stiff to very stiff clays with low-medium plasticity, medium silts with slight plasticity, and stiff to very stiff silts with low plasticity, respectively. These were the types of materials observed during the site surveys; therefore, the results obtained are consistent with literature reports as well as with field surveys. This new framework may be useful to provide real-time estimates of the drilled soil’s parameters, as well as updating CFAPs designs during their execution. This way, sustainable designs can be achieved, where substrata materials are better characterized, avoiding over-designed structures.

Interaction Modelling of High Dimensional Production Data

The final goal of this study is to present a practical interaction modeling methodology for big and complex industrial data in which there are mixed difficulty of high dimension, sparsity, interaction, multicollinearity, series bias, low signal noise ratio, nonlinearity, and so on. Particularly, this paper focuses on the issues of high dimension, sparsity, and interactions. In addition, it is necessary to estimate unknown factors and interactions in order to improve overall efficiency in complex and diverse situations of industry. The main purpose here is to summarize the applicability and challenges of Factorization Machines (FMs) to manufacturing data. FMs has possibility because of the ad-vantage of to identify unknown factors and its robustness to data with more than 95% missing values which has already been reported for recommendation problems. Manufacturing data assumed here incudes the inspected quality of the final products, and a set of various variables (settings, sensing results, etc.) of the series production process which may relate to the quality. Common characteristics of manufacturing data include mixture of numerical and categorical variables, high dimensionality, many missing values, and so on. Therefore, this paper particularly focuses on sparse high-dimensional type-mix data. So far, while interaction modeling of high-dimensional data has been discussed and verified, mainly for screening methods, still there is very few studies to apply FMs to the manufacturing data with many numerical variables. There-fore, we report the evaluation results using some synthesis data and simulation data from actual oracle model to summarize FMs applicability as the first step. The second step is to compare the performance of sparse FMs which has feature selection function, beside our proposal.

Big Data Technology Architecture Proposal for Smart Agriculture for Moroccan Fish Farming

As the global population increases rapidly, so does the need for fishing products. Aquaculture is well-developed in Asian countries but is underdeveloped in countries that share Morocco's climate. To meet the rising demands for aquaculture production, it is vital to embrace new digital strategies to manage the massive amount of data generated by the aquaculture environment. By employing Big Data methodologies, aquaculture activity is handled more effectively, resulting in increased production and decreased waste. This phase enables fish farmers and academics to obtain valuable data, increasing their productivity. Although Big Data approaches provide numerous benefits, they have yet to be substantially implemented in agriculture, particularly in fish farming. Numerous research projects investigate the use of Big Data in agriculture, but only some offer light on the applicability of these technologies to fish farming. In addition, no research has yet been undertaken for the Moroccan use case. This study aims to demonstrate the significance of investing in aquaculture powered by Big Data. This study provides data on the situation of aquaculture in Morocco in order to identify areas for improvement. The paper then describes the adoption of Big Data technology to intelligent fish farming and proposes a dedicated architecture to address the feasibility of the solution. In addition, methodologies for data collecting, data processing, and analytics are highlighted. This article illuminates the possibilities of Big Data in the aquaculture business. It demonstrates the technological and functional necessity of incorporating Big Data into traditional fish farming methods. Following this, a concept for an intelligent fish farming system based on Big Data technology is presented.