Big Data Methodologies Research Articles

Modeling Health Data Using Machine Learning Techniques Applied to Financial Management Predictions

Health management has steadily improved in performance and accuracy using IT technology. Hospitals and health institutions hold an enormous number of data in their software applications, which can be used with Big Data methodologies to extract useful information. One of the most challenging aspects of health institutional management is financial management; billing prediction is a key aspect to maintain a predictable service level for patients, avoiding unpleasant surprises and anticipating treasury management. Using patient data from public patient databases and applying a machine learning approach, this article offers a model that helps to make more precise and detailed financial plans.

Towards Developing a Robust Intrusion Detection Model Using Hadoop-Spark and Data Augmentation for IoT Networks.

In recent years, anomaly detection and machine learning for intrusion detection systems have been used to detect anomalies on Internet of Things networks. These systems rely on machine and deep learning to improve the detection accuracy. However, the robustness of the model depends on the number of datasamples available, quality of the data, and the distribution of the data classes. In the present paper, we focused specifically on the amount of data and class imbalanced since both parameters are key in IoT due to the fact that network traffic is increasing exponentially. For this reason, we propose a framework that uses a big data methodology with Hadoop–Spark to train and test multi-class and binary classification with one-vs-rest strategy for intrusion detection using the entire BoT IoT dataset. Thus, we evaluate all the algorithms available in Hadoop–Spark in terms of accuracy and processing time. In addition, since the BoT IoT dataset used is highly imbalanced, we also improve the accuracy for detecting minority classes by generating more datasamples using a Conditional Tabular Generative Adversarial Network (CTGAN). In general, our proposed model outperforms other published models including our previous model. Using our proposed methodology, the F1-score of one of the minority class, i.e., Theft attack was improved from 42% to 99%.

Prevalencia de cáncer en pacientes con hipotiroidismo: análisis mediante herramientas de big data

ObjectiveTo evaluate the frequency of different types of cancer in patients diagnosed with hypothyroidism using big data methodology on the Savana Manager platform. MethodsAn observational, retrospective study was carried out using electronic medical record (EMR) data from the Hospital Universitario Puerta de Hierro Majadahonda (Madrid). Information from the EMRs was extracted using artificial intelligence techniques and analysed using the Savana Manager v3.0 software. Searches were performed using the term “hypothyroidism” and the terms corresponding to the tumours analysed. ResultsOf a total of 506,749 patients studied, 23,570 (4.7%) were diagnosed with hypothyroidism. Patients with this diagnosis had a significantly higher frequency of cancer than that found in non-hypothyroid subjects (OR 2.09, 95% confidence interval [CI] 2.01-2.17). This higher frequency was found both in women (OR 1.99, 95% CI 1.90-2.08) and in men (OR 2.83, 95% CI 2.63-3.05). However, this higher frequency of cancer was not observed in hypothyroid patients older than 60 years (OR 0.97, 95% CI 0.92-1.02). Although the frequency of most of the neoplasms studied individually was higher in those with hypothyroidism, we observed that hypothyroid patients over 60 years of age had a significant decrease in the frequency of prostate, lung, colorectal and liver cancer. ConclusionData from this hospital cohort suggest that there is a significant association between a diagnosis of hypothyroidism and cancer. However, this association is less evident in hypothyroid patients older than 60 years.

Analysis of Big Data Methodology for Pattern Recognition

Abstract: Sentiment Analysis (SA) is an ongoing field of research in text mining field. SA is the computational treatment of opinions, sentiments and subjectivity of text. This survey paper tackles a comprehensive overview of the last update in this field. Many recently proposed algorithms' enhancements and various SA applications are investigated and presented briefly in this survey. These articles are categorized according to their contributions in the various SA techniques. The related fields to SA (transfer learning, emotion detection, and building resources) that attracted researchers recently are discussed.

Normalizing player surveillance through video game infographics

As video game production is becoming increasingly data-driven, player surveillance shapes the everyday realities of users and developers. Remote online tracking and the resulting optimization and governance of in-game activity subscribe to the Big Data methodology as a way of accounting for entire player populations. By design, player surveillance serves the interests of developers and publishers, who have exclusive access to this proprietary data. Yet, discursively, these parties attempt to present surveillance as a mutually beneficial endeavor aimed at improving video games. A part of this strategy is the video game industry’s selective information disclosure, which I explore empirically on the example of telemetry infographics. Based on a thematic analysis of 200 infographics from 127 games, I show how publicly disseminated infographics contribute to the normalization of player surveillance by presenting it as a source of harmless trivia to be collected and shared by fans and the specialized press.

Semiotic approaches to big data visualization

With a few exceptions – focusing on journalism (Compagno 2017), on works of art (Chartier, Pulizzotto, Chartrand & Meunier 2019, Dondero 2020), on deep fake videos (Leone 2021, Dondero 2021), and on data epistemology (Bachimont 2018) – semiotics has been late to approach big data. In contrast to information and communication studies, which have been quick to recognize the big data revolution in medical, biological, political, urbanistic, and journalistic practices, and while digital humanities and digital art history continue to offer crucial insight with their analysis of large collections of artworks and heritage objects, semiotics has yet to live up to the big data challenge. In this Punctum issue we have attempted to (1) contribute to the global discussion of possible semiotic approaches to big data theory and methodology; (2) explore specifically an epistemological approach to visualizations of big data, i.e., to their manipulation, design, display, and interpretation; (3) study, from a critical vantage point, the ideologies and epistemic perspectives that underlie the acts of collecting and visualizing big data, prior to their analysis. Our overall objective has been to trace the path that leads from visualization, regarded as a unifying representation of disparate data, to its full elevation to an interpretative device.

A value based approach to assessing changes in cultural policies

ABSTRACT In democratic societies, the political process ultimately is about making value judgements. Comparative cultural policy research indicates different value orientations may be behind cultural policies of different nations. It has also indicated developments that are common, such as democratisation of culture, neoliberalisation of politics and globalisation. As of yet, such value changes have not been studied at the level of cultural policy documents analysed as text [Bell, D., & Oakley, K. (2014). Cultural policy. Routledge]. This article proposes a methodology to study value changes behind cultural policies over time and across nations. Based in the “pragmatic sociology” of Boltanski, Thévenot, and Chiapello [(2005). The new spirit of capitalism. (Translated by Gregory Elliot from Le nouvel esprit du capitalisme). Gallimard Verso. (Original work published 1999); (2006). On justification, economies of worth. (Translated by Catherine Porter from De la justification, Les économies de la grandeur). Gallimard & Princeton University Press. (Original work published 1991)], the article presents a big data methodology to uncover value changes of policy documents over time, between different policy agents, and across nations using material from the UK and the Netherlands. The paper explains the methodology, discusses the implications of the preliminary outcomes and provides suggestions for further comparative policy analyses.

Overcoming ‘confirmation bias’ and the persistence of conspiratorial types of thinking

ABSTRACT Discussions on media and fakery are usually premised on the general public being manipulated by mainstream media bias or fabrications emanating from the Internet. It is less common in the discipline of media and communication to speculate about users’ reasons for accepting what appear to be basic untruths: I will suggest here that discussions about users’ complicity must become more central to our attempts to understand media and fakery. Rather than a simple rebuttal of the ‘facts’ or the promotion of big data methodologies, this paper will suggest that deploying convincing counter-narratives are a better means of convincing those we suspect of being susceptible to confirmation bias and conspiratorial types of thinking that there are better ways of understanding contemporary politics.

Social media data reveals multiple cultural services along the 8.500 kilometers of Brazilian coastline

Cultural ecosystem services (CES) are benefits that people receive from ecosystems, for example, through spiritual enrichment, cognitive development, recreation, and aesthetic experiences. These are important contributors to human well-being, but are challenging to measure due to their intangible nature and because they may vary spatially depending on ecosystem condition and restrictions on use. Here, we employ a big data methodology to identify CES over a large scale case study with the purpose of providing insights for marine and coastal management. First, we used machine learning to identify features present in 21,789 Flickr photographs taken across approximately 8,500 km of the Brazilian coastline. Then, we associated the keywords describing the identified features with broad CES categories allowing us to identify and map the geographical distribution of cultural services. We found that CES related to aesthetic experiences were more represented in photographs taken in protected areas, while other forms of cultural value (e.g. sport recreation, social recreation) were more frequent in unprotected areas. Notwithstanding the unavoidable biases in favour of certain CES representations, our results illustrate diverse forms of cultural services generated by the Brazilian coast. While there was a generally higher provision of CES per user in protected areas reflecting their enhanced potential to support tourism and other CES-related activities, the image classification algorithm was not able to identify CES which are less dependent on the biophysical domain such as spiritual values, inspiration, education and arts. We believe that our approach can be applied at broader scales (continental and global) in order to uncover cultural services in multiple environments while strongly recommending integration with social science based methodologies and expertise. We further advocate for the employment of such automatic evaluation at local scale as a tool to complement other approaches in support of protected area management, including resource allocation and investments to potentialize CES (i.e. recreational infrastructure, wildlife observation posts) or other relevant ecosystem services.

Examining geographical disparities in the incubation period of the COVID-19 infected cases in Shenzhen and Hefei, China

BackgroundCurrent studies on the COVID-19 depicted a general incubation period distribution and did not examine whether the incubation period distribution varies across patients living in different geographical locations with varying environmental attributes. Profiling the incubation distributions geographically help to determine the appropriate quarantine duration for different regions.MethodsThis retrospective study mainly applied big data analytics and methodology, using the publicly accessible clinical report for patients (n = 543) confirmed as infected in Shenzhen and Hefei, China. Based on 217 patients on whom the incubation period could be identified by the epidemiological method. Statistical and econometric methods were employed to investigate how the incubation distributions varied between infected cases reported in Shenzhen and Hefei.ResultsThe median incubation period of the COVID-19 for all the 217 infected patients was 8 days (95% CI 7 to 9), while median values were 9 days in Shenzhen and 4 days in Hefei. The incubation period probably has an inverse U-shaped association with the meteorological temperature. The warmer condition in the winter of Shenzhen, average environmental temperature between 10 °C to 15 °C, may decrease viral virulence and result in more extended incubation periods.ConclusionCase studies of the COVID-19 outbreak in Shenzhen and Hefei indicated that the incubation period of COVID-19 had exhibited evident geographical disparities, although the pathological causality between meteorological conditions and incubation period deserves further investigation. Methodologies based on big data released by local public health authorities are applicable for identifying incubation period and relevant epidemiological research.

Mortality in Belarus During the COVID-19 Pandemic

The author analyzes the influence of demographic factors on the spread of the coronavirus pandemic. The basis of the analysis is the use of the Big Data methodology. The influence and consequences of urbanization, migration, demographic structure, and information sources on the state and prospects of mortality are considered. The article provides an analysis of the state of mortality processes in the Republic of Belarus for the period 1990-2019, and shows the influence of the coronavirus pandemic on the state of mortality processes and demographic losses in the Republic of Belarus in 2020. The association of the spread of COVID-19, migration and its consequences with the process of urbanization and the growth of agglomerations is revealed. The article formulates the position on the dominance of various types of migration in the spread of viral diseases. Various types of international migration are the carriers of the covid pandemic. The article describes the relationship of the Belarusian labor migration to the spread of the COVID-19 pandemic in the country. Within the country, the carriers of the virus and the magnitude of the incidence are associated with the pendulum and, above all, with labor migration. The article presents comparable information on the spread of the COVID-19 pandemic in the Republic of Belarus in comparison with the countries bordering it – Russia, Ukraine, Poland, Lithuania and Latvia. The article substantiates the position on the influence of changes in the age structure of the population on the dynamics of total mortality. The influence of the coronavirus pandemic on the structure of causes of death was studied. The obvious and latent consequences of the coronavirus pandemic on demographic processes and their underlying factors are revealed. The conclusion is made about the trends and the state of total mortality in Belarus in the future.

True Liquidity and Fundamental Prices: US Tick Size Pilot

We develop a big-data methodology to estimate fundamental prices and true liquidity measures, explicitly considering the rounding specification due to the minimum tick size. Evaluation of the tick size pilot (TSP), which increased the tick size for some randomly chosen stocks, requires the impact of rounding. Our true liquidity measures capture the TSP-driven decreased inventory costs of market-makers, whereas traditional measures without the rounding adjustment cannot. We nd that the TSP increases market-maker pro ts, but does not improve liquidity and price efficiency. This result contrasts existing empirical studies but is consistent with recent theoretical studies that account for rounding.

Clinical Management of COPD in a Real-World Setting. A Big Data Analysis

ObjectiveThe aim of this study was to evaluate the quality of diagnosis and treatment of COPD using Big Data methodology on the Savana Manager 2.1 clinical platform. Materials and methodsA total of 59,369 patients with a diagnosis of COPD were included from a population of 1,219,749 adults over 40 years of age. ResultsIn total, 78% were men. Spirometry data were available for only 26,453 (43.5%) subjects. Disease severity was classified in 18,172 patients: 4396 mild, 7100 moderate, and 6676 severe, although only 27%, 34%, and 28%, respectively, presented obstructive spirometry. The clinical management of COPD is mainly the responsibility of the primary care and pulmonology departments, while internal medicine and, to a lesser extent, geriatrics also participate. Drug treatment was based on bronchodilators and inhaled corticosteroids (ICS). A marked decline in the use of long-acting beta-2 agonists (LABA) in monotherapy and a slight reduction in ICS/LABA combinations, associated with a long-acting anticholinergic (LAMA) in 74% of cases, were observed. All-cause in-hospital mortality among the overall population was 5.6% compared to 1% of the general population older than 40 years. In total, 35% were admitted to hospital, with an average stay of 6.6 days and an in-hospital mortality rate in this group of 10.74%. DiscussionThis study identifies the main features of an unselected COPD population and the main errors made in the management of the disease.

Tracking Fluctuations in Psychological States using Social Media Language: A Case Study of Weekly Emotion

Personality psychologists are increasingly documenting dynamic, within–person processes. Big data methodologies can augment this endeavour by allowing for the collection of naturalistic and personality–relevant digital traces from online environments. Whereas big data methods have primarily been used to catalogue static personality dimensions, here we present a case study in how they can be used to track dynamic fluctuations in psychological states. We apply a text–based, machine learning prediction model to Facebook status updates to compute weekly trajectories of emotional valence and arousal. We train this model on 2895 human–annotated Facebook statuses and apply the resulting model to 303 575 Facebook statuses posted by 640 US Facebook users who had previously self–reported their Big Five traits, yielding an average of 28 weekly estimates per user. We examine the correlations between model–predicted emotion and self–reported personality, providing a test of the robustness of these links when using weekly aggregated data, rather than momentary data as in prior work. We further present dynamic visualizations of weekly valence and arousal for every user, while making the final data set of 17 937 weeks openly available. We discuss the strengths and drawbacks of this method in the context of personality psychology's evolution into a dynamic science. © 2020 European Association of Personality Psychology

BIG DATA ANALYSIS IN HEALTH CARE DOMAIN: A SYSTEMATIC REVIEW

As the Volume of the data produced is increasing day by day in our society, the exploration of big data in healthcare is increasing at an unprecedented rate. Now days, Big data is very popular buzzword concept in the various areas. This paper provide an effort is made to established that even the healthcare industries are stepping into big data pool to take all advantages from its various advanced tools and technologies. This paper provides the review of various research disciplines made in health care realm using big data approaches and methodologies. Big data methodologies can be used for the healthcare data analytics (which consist 4 V’s) which provide the better decision to accelerate the business profit and customer affection, acquire a better understanding of market behaviours and trends and to provide E-Health services using Digital imaging and communication in Medicine (DICOM).Big data Techniques like Map Reduce, Machine learning can be applied to develop system for early diagnosis of disease, i.e. analysis of the chronic disease like- heart disease, diabetes and stroke. The analysis on the data is performed using big data analytics framework Hadoop. Hadoop framework is used to process large data sets Further the paper present the various Big data tools , challenges and opportunities and various hurdles followed by the conclusion.

Cost Effectiveness Analysis and Payment Policy Recommendation-Population-Based Survey with Big Data Methodology for Readmission Prevention of Patients with Paroxysmal Supraventricular Tachycardia treated with Radiofrequency Catheter Ablation.

Recurrence of paroxysmal supraventricular tachycardia (PSVT) has been reported to be lower in patients treated with radiofrequency catheter ablation (RFCA) than in those who are not. Few population-based surveys have stated the cost-effectiveness related to this treatment. We, therefore, performed a nationwide retrospective study using National Health Insurance Research Database (NHIRD) data from 2001–2012 in Taiwan. The incidence of PSVT-related admissions was computed from patients’ first admission for a primary PSVT diagnosis. There were 21,086 patients hospitalized due to first-time PSVT, of whom 13,075 underwent RFCA, with 374 recurrences (2.86%). In contrast, 1751 (21.86%) of the remaining 8011 patients who did not receive RFCA, most of whom had financial concerns, experienced PSVT recurrence. The relative PSVT recurrence risk in those who did not receive RFCA was 7.6 times (95% CI: 6.67–8.33) that of those who did undergo RFCA. In conclusion, the PSVT recurrence rate was much higher in patients who did not receive RFCA at their first admission. Furthermore, RFCA proved cost-effective, with the ratio of the incremental cost-effectiveness ratio (ICER) and gross domestic product (GDP) being only 1.15. To prevent readmission and avoid incremental cost, the authority could provide a financial supplement for every patient so that the procedure is performed, reducing the PSVT-recurrence life-years (disease-specific DALY).

Evaluating graph resilience with tensor stack networks: a Keras implementation

In communication networks resilience or structural coherency, namely the ability to maintain total connectivity even after some data links are lost for an indefinite time, is a major design consideration. Evaluating resilience is a computationally challenging task since it often requires examining a prohibitively high number of connections or of node combinations, depending on the structural coherency definition. In order to study resilience, communication systems are treated in an abstract level as graphs where the existence of an edge depends heavily on the local connectivity properties between the two nodes. Once the graph is derived, its resilience is evaluated by a tensor stack network (TSN). TSN is an emerging deep learning classification methodology for big data which can be expressed either as stacked vectors or as matrices, such as images or oversampled data from multiple-input and multiple-output digital communication systems. As their collective name suggests, the architecture of TSNs is based on tensors, namely higher-dimensional vectors, which simulate the simultaneous training of a cluster of ordinary multilayer feedforward neural networks (FFNNs). In the TSN structure the FFNNs are also interconnected and, thus, at certain steps of the training process they learn from the errors of each other. An additional advantage of the TSN training process is that it is regularized, resulting in parsimonious classifiers. The TSNs are trained to evaluate how resilient a graph is, where the real structural strength is assessed through three established resiliency metrics, namely the Estrada index, the odd Estrada index, and the clustering coefficient. Although the approach of modelling the communication system exclusively in structural terms is function oblivious, it can be applied to virtually any type of communication network independently of the underlying technology. The classification achieved by four configurations of TSNs is evaluated through six metrics, including the F1 metric as well as the type I and type II errors, derived from the corresponding contingency tables. Moreover, the effects of sparsifying the synaptic weights resulting from the training process are explored for various thresholds. Results indicate that the proposed method achieves a very high accuracy, while it is considerably faster than the computation of each of the three resilience metrics. Concerning sparsification, after a threshold the accuracy drops, meaning that the TSNs cannot be further sparsified. Thus, their training is very efficient in that respect.

Manejo clínico de la EPOC en situación de vida real. Análisis a partir de big data

ObjectiveThe aim of this study was to evaluate the quality of diagnosis and treatment of COPD using Big Data methodology on the Savana Manager 2.1 clinical platform. Materials and methodsA total of 59,369 patients with a diagnosis of COPD were included from a population of 1,219,749 adults over 40 years of age. ResultsIn total, 78% were men. Spirometry data were available for only 26,453 (43.5%) subjects. Disease severity was classified in 18,172 patients: 4,396 mild, 7,100 moderate, and 6,676 severe, although only 27%, 34%, and 28%, respectively, presented obstructive spirometry.The clinical management of COPD is mainly the responsibility of the primary care and pulmonology departments, while internal medicine and, to a lesser extent, geriatrics also participate. Drug treatment was based on bronchodilators and inhaled corticosteroids (ICS). A marked decline in the use of long-acting beta-2 agonists (LABA) in monotherapy and a slight reduction in ICS/LABA combinations, associated with a LAMA in 74% of cases, was observed.All-cause in-hospital mortality among the overall population was 5.6% compared to 1% of the general population older than 40 years. In total, 35% were admitted to hospital, with an average stay of 6.6 days and a rate of hospital mortality in this group of 10.74%. DiscussionThis study identifies the main features of an unselected COPD population and the major errors made in the management of the disease.

Automatic classification of dental artifact status for efficient image veracity checks: effects of image resolution and convolutional neural network depth

Enabling automated pipelines, image analysis and big data methodology in cancer clinics requires thorough understanding of the data. Automated quality assurance steps could improve the efficiency and robustness of these methods by verifying possible data biases. In particular, in head and neck (H&N) computed-tomography (CT) images, dental artifacts (DA) obscure visualization of structures and the accuracy of Hounsfield units; a challenge for image analysis tasks, including radiomics, where poor image quality can lead to systemic biases. In this work we analyze the performance of three-dimensional convolutional neural networks (CNN) trained to classify DA statuses. 1538 patient images were scored by a single observer as DA positive or negative. Stratified five-fold cross validation was performed to train and test CNNs using various isotropic resampling grids (643, 1283 and 2563), with CNN depths designed to produce 323, 163, and 83 machine generated features. These parameters were selected to determine if more computationally efficient CNNs could be utilized to achieve the same performance. The area under the precision recall curve (PR-AUC) was used to assess CNN performance. The highest PR-AUC (0.92 ± 0.03) was achieved with a CNN depth = 5, resampling grid = 256. The CNN performance with 2563 resampling grid size is not significantly better than 643 and 1283 after 20 epochs, which had PR-AUC = 0.89 ± 0.03 (p -value = 0.28) and 0.91 ± 0.02 (p -value = 0.93) at depths of 3 and 4, respectively. Our experiments demonstrate the potential to automate specific quality assurance tasks required for unbiased and robust automated pipeline and image analysis research. Additionally, we determined that there is an opportunity to simplify CNNs with smaller resampling grids to make the process more amenable to very large datasets that will be available in the future.

How Does Spectrum Affect Mobile Network Deployments? Empirical Analysis Using Crowdsourced Big Data

Despite the growing trend towards the use of big data methodologies, there is still limited application of such techniques to understand how spectrum is used in mobile networks. In this paper we analyse how low (<; 1 GHz) and high (>1 GHz) frequency spectrum is used in 4G networks in urban areas, in relation to eNodeB density, available bandwidth, Reference Signal Received Power (RSRP) and Reference Signal Received Quality (RSRQ). We present a method to analyse the strategies used by Mobile Network Operators (MNOs) to deal with traffic congestion, and the degree to which they must densify their networks depending on their spectrum portfolio. Using crowdsourced data from 2017 from a popular mobile app, we apply this method to Greater London. We find that the fraction of sites that fully use all available bands to the MNO range from 2% to 20%. Additionally, MNOs with large bandwidth use 42% fewer sites on average in dense urban environments. This difference decreases in suburban areas to 23% fewer sites. The lowest frequencies in each eNodeB tend to exhibit lower RSRP values, as they are often used to serve cell-edge users. These frequencies also show lower RSRQ values because of higher interference caused by neighbouring cells. Similarly, large (high frequency) bandwidth improves RSRQ as it allows for fewer users per MHz, which reduces interference and enables larger cell sizes. We conclude that in dense urban environments, the available bandwidth, rather than propagation properties, determines the preferred band for network deployment by MNOs.