Big Data Methods Research Articles

Advanced Big Data Mining Techniques for Early Detection of Heart Attacks in Clinical Data

Heart attacks are among the most dangerous ailments that people may develop. The key to controlling cardiovascular disease is to compare, contrast, and mine enormous volumes of data in a sequence that may be used to identify, control, and treat persistent problems, such as heart attacks. Forecasting, preventing, monitoring, and diagnosing cardiovascular diseases may be done through huge efficiency via big data analytics, which is well-known in the business sector for its useful application in regulating, comparing, and supervising enormous datasets. Big data technologies or methods used to mine massive databases for information include Hadoop, data mining, and visualization. Those fresh ideas, which have a wide range of uses, might be helpful in several industries, include medical. In this paper, we extend a big data mining pattern using a machine learning method to forecast the frequency of heart attacks from medical databases. Data preprocessing using the z- score normalization and feature extraction using Linear Discriminant Analysis (LDA) and classification using the Improved Random Forest (IRF). We generate enhanced presentation intensity with accuracy, precision, recall, and F- measure throughout the forecast model for heart disease with the IRF.

Robust Benchmark for Propagandist Text Detection and Mining High-Quality Data

Social media, fake news, and different propaganda strategies have all contributed to an increase in misinformation online during the past ten years. As a result of the scarcity of high-quality data, the present datasets cannot be used to train a deep-learning model, making it impossible to establish an identification. We used a natural language processing approach to the issue in order to create a system that uses deep learning to automatically identify propaganda in news items. To assist the scholarly community in identifying propaganda in text news, this study suggested the propaganda texts (ProText) library. Truthfulness labels are assigned to ProText repositories after being manually and automatically verified with fact-checking methods. Additionally, this study proposed using a fine-tuned Robustly Optimized BERT Pre-training Approach (RoBERTa) and word embedding using multi-label multi-class text classification. Through experimentation and comparative research analysis, we address critical issues and collaborate to discover answers. We achieved an evaluation performance accuracy of 90%, 75%, 68%, and 65% on ProText, PTC, TSHP-17, and Qprop, respectively. The big-data method, particularly with deep-learning models, can assist us in filling out unsatisfactory big data in a novel text classification strategy. We urge collaboration to inspire researchers to acquire, exchange datasets, and develop a standard aimed at organizing, labeling, and fact-checking.

Big data methods and news communication innovation from theoretical definition to operational route

Abstract To make news communication better adapt to the era of big data, it is necessary to innovate news communication. Accordingly, this paper constructs a big data analysis platform for news communication innovation methods based on the regression analysis model in big data analysis, determines the parameters and coefficients of the regression model with the help of the maximum likelihood estimation method, and determines the form of the regression model. Based on the platform, the main paths of news communication innovation are analyzed, standardized regression coefficients are calculated, and the main methods of news communication innovation are changing news communication forms and news communication media. To verify that the above methods can make news communication innovative, we designed a visualization information platform to change the news communication format and launched a simulation experiment to analyze the effect of news communication under different media. The results showed that after the visualization of the news communication format, the number of viewers of news programs increased from 2.45 million to 3.65 million, an increase of 1.2 million. The conversion rate of audience behavior of news programs increased from the original 63% to 87%, an increase of 24%. From the above results, it is clear that news communication under the big data approach has improved from a theoretical definition to an operational route.

Compressed computations using wavelets for hidden Markov models with continuous observations.

Compression as an accelerant of computation is increasingly recognized as an important component in engineering fast real-world machine learning methods for big data; c.f., its impact on genome-scale approximate string matching. Previous work showed that compression can accelerate algorithms for Hidden Markov Models (HMM) with discrete observations, both for the classical frequentist HMM algorithms-Forward Filtering, Backward Smoothing and Viterbi-and Gibbs sampling for Bayesian HMM. For Bayesian HMM with continuous-valued observations, compression was shown to greatly accelerate computations for specific types of data. For instance, data from large-scale experiments interrogating structural genetic variation can be assumed to be piece-wise constant with noise, or, equivalently, data generated by HMM with dominant self-transition probabilities. Here we extend the compressive computation approach to the classical frequentist HMM algorithms on continuous-valued observations, providing the first compressive approach for this problem. In a large-scale simulation study, we demonstrate empirically that in many settings compressed HMM algorithms very clearly outperform the classical algorithms with no, or only an insignificant effect, on the computed probabilities and infered state paths of maximal likelihood. This provides an efficient approach to big data computations with HMM. An open-source implementation of the method is available from https://github.com/lucabello/wavelet-hmms.

Big Data Mining, digital tools, and methods to compare China and the West: The new agenda in global (economic) history

The paradigm of the new global (economic) history aims to present new case studies cross-referencing western and Asian sources and using new tools and methods from other disciplines such as digital humanities and computer sciences. Within such an interdisciplinary approach new digital solutions are being conceptualized and implemented to renew global history research and revise the Great Divergence debate. The GECEM Project Database is presented as example that follows the paradigm of the “Complex-Systems Landscape” metaphor used and developed in this special issue. Thus, the core hypothesis in these papers is based on the role and agency of merchants and the networks they formed which ultimately changed consumer behavior and fostered the global circulation of goods in early modern China, Europe, and the Americas.

A Big-Data-Based Experimental Platform for Green Shipping Monitoring and Its Teaching Application

The construction of New Business Studies (NBS) in China and big data technology offer an opportunity for teaching reform. Based on the existing teaching resources, professional knowledge, data, and technology, we monitored the dynamics and checked the statistics of air pollutant emissions from ships in global waters. Various techniques of big data analysis and methods of artificial intelligence were employed, including data collection, data fusion, feature analysis, deep learning network, and system testing. Specifically, the scenario of green shipping monitoring was reproduced by virtual reality; experimental learning was carried out, involving five experimental methods, eight experimental steps, and ten interactive operations; and the results of the experimental learning were assessed. In this way, the students had a better cognition of datasets, a deeper understanding of data correlation, and an improved mastery of interactive operations. In addition, the students varied in terms of learning performance, experimental participation, and active performance inspired by individual thinking. Overall, the students were satisfied with the quality of experimental learning.

Bigdata Technique Implementation in Bank Industry: Evidence from China

With the continuous maturity and progress of cloud computing, intelligent tools, Internet of Things technology and other technologies, society has entered the era of big data, which is both a challenge and an opportunity for all traditional industries. As the pillar of national economy, the financial industry should comply with the trend of the time, comprehensively analyze the impact, use big data to analyze and mine effective information and reasonably apply it to the operation of banks. Finding the path of change is the key to foster the synchronous and long-term development of the banking industry in the new era. To achieve the aim of promoting the innovation and reform of the banking industry effectively, it’s necessary to dig the answers from the examples of big data application. This study introduces the basic concepts and analysis methods of big data, shows the typical application of big data in Chinese banks, mines the convergence between banking and big data, analyzes the existing problems of big data finance and puts forward solutions. In addition, this research looks forward to the future intelligent and regular big data technique implementation in the financial industry. Overall, these results shed light on guiding further exploration of the big data implementation in financial industry.

The Intersections between Sociology and STS: A Big Data Approach

This paper charts the changing intersections between sociology and science and technology studies (STS) using computational textual analysis. We characterize this “quali-quantitative” approach as a Big Data method, as this calls attention to the commixture of textual and numeric data that characterizes Big Data. The term Big Data, too, calls attention to the increasing privatization of both data and data analytics tools. The data mining was done using a commercial analytics tool, IBM SPSS Modeler, that to the best of our knowledge has not yet been used for STS or sociological research. The identification of intersections occurred as part of a larger project to analyze political-economic and epistemic changes within STS, focusing on academic publishing. These epistemic changes were identified qualitatively, through 76 interviews with STS scholars, and quantitatively, through a computational analysis of three decades of STS journals (1990–2019).

Research on the Fusion of Power Big Data Technology and Power System Simulation Calculation

Power big data is the practice of big data concepts, methods, and technologies in the power industry. It is composed of structured data and unstructured data. It mainly involves power generation, transmission, transformation, and distribution. Power big data can reflect the macro-economy in real time, promote economic and social development, and improve the internal management level and economic benefits of the power industry enterprises. The integration of power big data technology and power system simulation calculation can update system analysis in real-time and provide new perspectives and methods for system analysis.

Market Analysis for Farmers using Machine Learning

Using big data Data mining and predictive analysis have developed to be helpful for examining crop prices in agriculture. This paper considers applying methods of Big Data and data mining utilising agricultural data. As a result of an outdated chain structure for selling the crops in their various markets, farmers are increasingly concerned with getting the full worth of the commodity. Due to the fact that farmers in the area may only sell their goods in a way that indirectly affects consumers, the market's true potential is still untapped. E-agriculture is a growing industry that focuses on improving farming and advancing provincial development through better data and correspondence forms. The market has a growing amount of data, just as other industries. Data from the various markets in the Bangalore area are taken into account and employed in this work's data mining decision tree models. Farmers are the backbone of our nation, which is dealing with a number of issues. Among these issues, getting pertinent information about the crop and weather specifics at the appropriate moment is one of the most significant. The primary objective of the Map-Reduce-based decision tree model is to give Farmers the greatest price for their product. In comparison to other models, the output of the proposed Map Reduce model is more accurate at predicting the best price from the existing market data.

A Personalized Learning Path for French Study in Colleges Based on a Big Data Knowledge Map

The education industry is gradually improving with the rapid development of information technology. The learners use networks and computers to alter the traditional instructional framework based on educational information technology and achieve personalized learning. This teaching method emphasizes each learner’s identity and autonomy. However, due to the huge number of learning resources available on the Internet, students lack relevant courses, clear learning tasks, and the connection between various knowledge points, resulting in an unsatisfactory effect on the learning process. Knowledge maps for different learner types are created using historical learners’ conceptual knowledge and the segmentation and correlation technique of big data knowledge maps. Using a big data method in this process will automatically generate a set of weak conceptual learning pathways. For this problem, in the era of big data, people put forward the concept of knowledge map and used the algorithm based on the big data knowledge map to study the personalized learning path for college French. The content, structure, and relationship of college French knowledge points can be accurately expressed using this method, which is preferred by college administrators and teachers. This paper aims to investigate the personalized learning path for college French using a big data knowledge map, starting with the characteristics of a college French field of study. This study provides technical support in the establishment of a big data knowledge map based on a learning path recommendation framework. So, after the performance of several commonly used learning path recommendation algorithms, three French students have been selected at random for learning path planning. The results show that personalized learning path planning can be realized based on a knowledge map pre-repair relationship and objective attributes. In the analysis, not only the proposed technique is compared with the conventional optimization approach, but also a comparison study on the benefits of several learning effect prediction models is also performed. The results of this study suggest that this algorithm has a high learning efficiency and that the effective implementation of recommendations produced using our proposed strategy has a significant advantage.

ArcticBirdSounds: An open-access, multiyear, and detailed annotated dataset of bird songs and calls.

Tracking biodiversity shifts is central to understanding past, present, and future global changes. Recent advances in bioacoustics and the low cost of high-quality automatic recorders are revolutionizing studies in biogeography and community and behavioral ecology with a robust assessment of phenology, species occurrence, and individual activity. This large volume of acoustic recordings has recently generated a plethora of datasets that can now be handled automatically, mostly via big data methods such as deep learning. These approaches need high-quality annotations to classify and detect recorded sounds efficiently. However, very few strongly annotated datasets-that is, with detailed information on start and end time of each vocalization-are openly accessible to the public. Moreover, these datasets mostly cover temperate species and are usually limited to a single year of recordings. Here, we present ArcticBirdSounds, the first open-access, multisite, and multiyear strongly annotated dataset of arctic bird vocalizations. ArcticBirdSounds offers 20 h of annotated recordings over 2 years (2018, 2019), taken from 15 distinct plots within six locations across the Arctic, from Alaska to Greenland. Recordings cover the arctic vertebrates' breeding period and are evenly spaced during the day; they capture most species breeding there with 12,933 temporal annotations in 49 classes of sounds. While these data can be used for many pressing ecological questions, it is also a unique resource for methodological development to help meet the challenges of fast ecosystem transformations such as those happening in the Arctic. All data, including audio files, annotation files, and companion spreadsheets, are available in an Open Science Framework repository published under a CC BY 4.0 License.

COVID-19 contact tracking based on person reidentification and geospatial data

Efficient contact tracing is a crucial step in preventing the spread of COVID-19. However, the current methods rely heavily on manual investigation and truthful reporting by high-risk individuals. Mobile applications and Bluetooth-based contact tracing methods have also been adopted, but privacy concerns and reliance on personal data have limited their effectiveness. To address these challenges, in this paper, a geospatial big data method that combines person reidentification and geospatial information for contact tracing is proposed. The proposed real-time person reidentification model can identify individuals across multiple surveillance cameras, and the surveillance data is fused with geographic information and mapped onto a 3D geospatial model to track movement trajectories. After real-world verification, the proposed method achieves a first accuracy rate of 91.56%, a first-five accuracy rate of 97.70%, and a mean average precision of 78.03% with an inference speed of 13 ms per image. Importantly, the proposed method does not rely on personal information, mobile phones, or wearable devices, avoiding the limitations of existing contact tracing schemes and providing significant implications for public health in the post-COVID-19 era.

Databases of the evolution of the microbiome and its drug susceptibility in astronauts and hermetic facility operators

When planning lunar bases and during long-term interplanetary (autonomous) flights, the issue of accumulation of pathogenic microbiota in conditions of closed inhabited objects is relevant. This question can be answered on the basis of a long observation period. When conducting microbiological examinations on the International Space Station, it was shown that one of the main features of the human-microorganisms ecological system in the environment of manned spacecraft is the periodic accumulation of the pathogenicity potential of microorganisms. These results were confirmed in isolation ground experiments and in the analysis of the SALYUT-6, 7 and MIR, International Space Station missions. Our work presents databases of microbiota and its antibiotic resistance of cosmonauts and hermetic facility operators at various stages of space flight, or isolation experiments. The databases are designed to assess the evolution of the population of microorganisms and their antibiotic resistance in various parts of the body of cosmonauts and operators of hermetic objects before, during and after a space flight and isolation experiments. Information about the microbiome of astronauts has been streamlined and entered into the database since 1970. The databases contain information about the microbiota in chronological order and its antibiotic resistance in cosmonauts and pressurized facility operators. Samples of various parts of the body were analyzed: forehead, chest, neck, ear, arm, armpit, groin, intestines, plaque, oral cavity, pharynx, tongue, nose, cheeks. Experimental samples of microbiota obtained from the International Space Station are stored at a deep freeze of -76 °С, are examined using a standard bacteriological method, molecular biology research methods and immunochemical methods. The information obtained makes it possible to assess the normal and dysbiotic state of the cosmonauts' microbiota. The databases are constantly updated. Thanks to the adapted database format, it is possible to use neural network and BIG DATA methods for analysis.

Centralising Qualitative Research in Big Data Methods Through Algorithmic Ethnography

Responding to the challenge for qualitative researchers to claim a central place in conversations about big data, analytics, datafication, data mining and the role of algorithms, this article describes a mixed-method research partnership focused on algorithmic ethnography. In the debates about the opacity of online algorithms, qualitative researchers typically advocate for access to code. This standard discourse centralises the technical aspects of big data and networked ethnographies. Instead, this article outlines a research methodology that analyses algorithmic discourses by working alongside the technical expertise of data scientists and utilizes the affordability of big data methods to do qualitative work. The potential for qualitative research skills to investigate the underlying technical processes that frame online social interactions is proposed as a way to place how people understand the world at the centre of big data research.

Big Data in Chronic Kidney Disease: Evolution or Revolution?

Digital information storage capacity and biomedical technology advancements in recent decades have stimulated the maturity and popularization of “big data” in medicine. The value of utilizing big data as a diagnostic and prognostic tool has continued to rise given its potential to provide accurate and insightful predictions of future health events and probable outcomes for individuals and populations, which may aid early identification of disease and timely treatment interventions. Whilst the implementation of big data methods for this purpose is more well-established in specialties such as oncology, cardiology, ophthalmology, and dermatology, big data use in nephrology and specifically chronic kidney disease (CKD) remains relatively novel at present. Nevertheless, increased efforts in the application of big data in CKD have been observed over recent years, with aims to achieve a more personalized approach to treatment for individuals and improved CKD screening strategies for the general population. Considering recent developments, we provide a focused perspective on the current state of big data and its application in CKD and nephrology, with hope that its ongoing evolution and revolution will gradually identify more solutions to improve strategies for CKD prevention and optimize the care of patients with CKD.

Systematic Analysis of Position-Data-based Key Performance Indicators

Abstract In the past 20 years, performance analysis in soccer has accumulated a wide variety of key performance indicators (KPI’s) aimed at reflecting a team’s strength and success. Thanks to rapidly advancing technologies and data analytics more sophisticated metrics, requiring high resolution data acquisition and big data methods, are developed. This includes many position-data-based KPI’s, which incorporate precise spatial and temporal information about every player and the ball on the field. The present study contributes to this research by performing a large-scale comparison of several metrics mainly based on player positions and passing events. Their association with team’s success (derived from goals scored) and team’s strength (estimated from pre-game betting odds) is analysed. The systematic analysis revealed relevant results for further KPI research: First, the magnitude of overall correlation coefficients was higher for relative metrics than for absolute metrics. Second, the correlation of metrics with the strength of a team is stronger than the correlation with the game success of a team. Third, correlation analysis with team strength indicated more positive associations, while correlation analysis with success is most likely confounded by the intermediate score line of a game and revealed more negative associations.

The analytics and applications of big data and machine learning

Economic and social activities in today's world are becoming increasingly digital. The Internet and the assistance of artificial intelligence technologies have led to massive amounts of data from these activities. These data come from different sources and in various forms, both structured and unstructured. Some data have large sample sizes, while there are also high-dimensional, enormous data in which the dimensionality of the explanatory variables surpasses the sample size. These enormous data sets are valuable, and they could drive a variety of economic activities. Big data production, machine learning, and statistics are deeply interrelated. This article discusses the concepts and methods of big data and machine learning from the characteristics of big data and the nature of machine learning. In particular, the four characteristics of big data are deeply analyzed and discussed. In addition, the analysis discusses the relationship between machine learning and big data. The article concludes with a summary and outlook of the whole article.

Research Progress of Tumor Big Data Visualization

Background: As the number of tumor cases significantly increases, so does the quantity of tumor data. The mining and application of large-scale data have promoted the development of tumor big data. Among them, the visualization methods of tumor big data can well show the key information in a large volume of data and facilitate the human brain to receive information. Therefore, tumor big data visualization methods are a key part of the development of tumor big data. Process: This paper first summarizes the connotation, sources, characteristics, and applications of tumor big data, and expounds the current research status of tumor big data visualization at home and abroad. Then, this paper focuses on four mainstream visualization presentation methods of tumor big data, namely the visualization of tumor spatiotemporal data, the visualization of tumor hierarchy and network data, the visualization of tumor text data, and the visualization of multidimensional tumor data, and gives specific application scenarios. After this, the paper introduces the advantages, disadvantages, and scope of the use of five data visualization websites and software that can be easily obtained by readers. Finally, this paper analyzes the problems existing in tumor big data visualization, summarizes the visualization methods, and proposes the future of tumor big data visualization.

High Rises and Housing Stress

Problem, research strategy, and findings The financialization of housing is a rapidly growing concern for planning researchers and policymakers, but the opacity of property ownership in most cities has hampered efforts to rigorously measure the phenomenon. Here we introduce a new approach based on big data methods. By combining web scraping of property assessment, business registry, and rental advertisement data, we reliably identified the networks of property ownership lurking behind anonymous numbered companies and established the extent of financialized rental housing ownership. We demonstrate the effectiveness of this approach with a quantitative case study of the financialization of rental housing in Montreal (Canada). Using spatial regression and clustering analyses, we found that there are two distinct types of financialized rental housing ownership in Montreal: one characterized by precarious and student tenants and another characterized by affluent tenants. In general, high proportions of financialized ownership are associated with higher levels of housing stress and dense housing typologies. Takeaway for practice By demonstrating meaningful differences in housing market outcomes across financialization status—which has not usually been readily accessible to either renters or planners—our findings show the importance of rental market information asymmetry. Planners should treat landlord data as one component of the information necessary to properly regulate a rental housing market. Municipalities should make property ownership information publicly accessible to facilitate public scrutiny of residential land use and more effective protection of tenant rights.