General Data Research Articles

Exploring unseen 3D scenarios of physics variables using machine learning-based synthetic data: An application to wave energy converters

This work uses machine learning to produce synthetic data of wave energy converters from time-expensive 3D simulations based on computational fluid dynamics models. The simulations to analyse the response of these systems to incoming waves are lengthy and computationally expensive to obtain. Here, we explore the use of a beta-VAE and a Principal Components-based adversarial autoencoder for generating new synthetic data. The compression plus the generation of synthetic data introduces an exceptionally fast surrogate model of the original simulation and delivers more samples of either dynamic viscosity and velocity fields, enlarging the design space. The new generated synthetic samples can have a speed up from 5 to 6 orders of magnitude. The new design space can be used to improve the prediction of dynamic viscosity given the velocity fields. The generative model has the potential to capture the transition and the new physical phenomena under extreme initial conditions.

Linking Repeated Subjective Judgments and ConvNets for Multimodal Assessment of the Immediate Living Environment

The integration of alternative data extraction approaches for multimodal data, can significantly reduce modeling difficulties for the automatic location assessment. We develop a method for assessing the quality of the immediate living environment by incorporating human judgments as ground truth into a neural network for generating new synthetic data and testing the effects in surrogate hedonic models. We expect that the quality of the data will be less biased if the annotation is performed by multiple independent persons applying repeated trials which should reduce the overall error variance and lead to more robust results. Experimental results show that linking repeated subjective judgements and Deep Learning can reliably determine the quality scores and thus expand the range of information for the quality assessment. The presented method is not computationally intensive, can be performed repetitively and can also be easily adapted to machine learning approaches in a broader sense or be transferred to other use cases. Following aspects are essential for the implementation of the method:•Sufficient amount of representative data for human assessment.•Repeated assessment trials by individuals.•Confident derivation of the effect of human judgments on property price as an approbation for further generation of synthetic data.

Cost-aware optimal resource provisioning Map-Reduce scheduler for hadoop framework

<p><span lang="EN-US">Distributed data processing model has been one of the primary components in the case of data-intensive applications; furthermore, due to advancements in technologies, there has been a huge volume of data generation of diverse nature. Hadoop map reduce framework is responsible for adopting the ease of deployment mechanism in an open-source framework. The existing Hadoop MapReduce framework possesses high makespan time and high Input/Output overhead and it mainly affects the cost of a model. Thus, this research work presents an optimized cost aware resource provisioning MapReduce model also known as the cost-effective resource provisioning MapReduce (CRP-MR) model. CRP-MR model introduces the two integrated approaches to minimize the cost; at first, this model presents the optimal resource optimization and optimal Input/Output optimization cleansing in the Hadoop MapReduce (HMR) scheduler. CRP-MR is evaluated considering the bioinformatics dataset and CRP-MR performs better than the existing model. </span></p>

Privacy Challenges and Solutions in Big Data Analytics: A Comprehensive Review

Abstract: Large data collections with a more intricate and diversified structure are referred to as "big data." These traits are typically associated with more challenges when it comes to storing, analyzing, and applying additional procedures or getting results. The technique of analyzing vast quantities of intricate data to uncover patterns or connections that are concealed is known as "big data analytics." Nonetheless, there is a clear conflict between big data's extensive use and its security and privacy. This paper examines privacy needs in big data, distinguishes between privacy and security, and focuses on privacy and security issues in large data. This study addresses the applications of privacy in business by utilizing established techniques like HybrEx, k-anonymity, T-closeness, and L-diversity. Many privacy-preserving techniques have been created for protection of privacy at various phases of the big data life cycle (such as data generation, storage, and processing). This work aims to explain the issues facing current privacy preservation strategies in big data and to give a comprehensive evaluation of them. This study also introduces new approaches to privacy preservation in big data, such as quick anonymization of massive data streams, identity-based anonymization, differential privacy, privacy preserving big data publishing, and hiding a needle in a haystack. This study discusses big data privacy and security issues in healthcare. A comparative analysis of some contemporary large data privacy approaches is also conducted. A comparative analysis of the latest big data privacy approaches is also conducted.

Intrusion Detection System Using Probabilistic Adaptive Learning

Abstract: Machine learning and deep learning techniques are widely used to evaluate intrusion detection systems (IDS) capable of rapidly and automatically recognizing and classifying cyber-attacks on networks and hosts. However, when destructive attacks are becoming more extensive, more challenges develop, needing a comprehensive response. Numerous intrusion detection datasets are publicly accessible for further analysis by the cybersecurity research community. The Intrusion Detection System (IDS) is an effective tool utilized in cybersecurity systems to detect and identify intrusion attacks. With the increasing volume of data generation, the possibility of various forms of intrusion attacks also increases. Feature selection is crucial and often necessary enhance performance. The structure of the dataset can impact the efficiency of the machine learning model. Furthermore, data imbalance can pose a problem, but sampling approaches can help mitigate it. With technological advancements, machine learning-based methods have emerged as the cornerstone of modern intrusion detection, enabling more precise identification of abnormal behaviors and potential intrusions by learning the patterns of normal network traffic. In response to these challenges, this paper introduces an innovative intrusion detection model that amalgamates the Probabilistic Adaptiv eLearning Network architecture.

Big data and AI in employment: The dual challenge of workforce replacement and protecting customer privacy in biometric data usage

The integration of Artificial Intelligence (AI) and Big Data is ushering in profound transformations across various industries, with biometric data usage standing out due to its deep implications for workforce dynamics and customer privacy. This review article critically examines the dual challenges presented by AI-driven automation and the extensive use of biometric data analytics, focusing on the resultant job displacement and escalating privacy concerns. Biometric technologies such as facial recognition, fingerprint identification, and voice analysis are increasingly deployed across sectors including finance, healthcare, and retail. These technologies aim to enhance security measures, improve user experience, and optimize operational efficiencies. However, they also bring to light substantial ethical dilemmas, particularly concerning the privacy of individuals and the security of the data being collected. The pervasive collection and analysis of biometric data can lead to invasive surveillance and profiling, exacerbating risks to personal privacy. Moreover, the use of AI in automating tasks that were traditionally performed by human workers is leading to significant shifts in employment structures. While AI can increase efficiency and reduce costs, it also raises the specter of widespread job displacement. This potential for automation-driven unemployment is especially pronounced in sectors that heavily utilize routine, repetitive tasks, posing critical socio-economic challenges. This article also explores the regulatory and technological frameworks currently in place, and those that are needed to address these challenges. The effectiveness of existing data protection laws, such as the General Data Protection Regulation (GDPR) in the European Union, and the California Consumer Privacy Act (CCPA) in the United States, is assessed in the context of AI and biometric data. We discuss the role of policy in shaping the ethical use of AI and protecting workers, along with the technological safeguards that could be implemented to secure biometric data and ensure privacy. By synthesizing insights from recent research, case studies, and expert analyses, this article provides a comprehensive overview of how AI and Big Data are reshaping the landscape of work and privacy. It critically discusses the need for a balanced approach that harnesses the benefits of technological advancements while safeguarding individual rights and employment security.

A simultaneous optimization of the seawater bittern and waste seashell recovery process for CO2 and SOx utilization

This study proposes a novel integration of seawater bittern and waste seashell recovery processes for CO2 and SOx utilization. Furthermore, simultaneous optimization of waste seashells and Ca(OH)2 separated from seawater bittern as an SOx absorbent was conducted to derive the blending ratio using a genetic algorithm (GA). The GA-based optimization process consists of four steps: (1) data generation using process model and data preprocessing, (2) DNN-based surrogate model development to predict the gypsum purity, CaCO3 yield, and CO2 emissions, (3) mathematical model development for TAC calculation, (4) GA-based optimization to derive the cost-optimal blending ratio and TAC. As a result, the derived cost-optimal blending ratios were scallop shells (0.03 %), cockle shells (0.03 %), clam shells (13.66 %), oyster shells (40.31 %), mussel shells (28.80 %), and Ca(OH)2 (17.18 %), reducing the TAC by up to 10.88 % compared to conventional CO2 and SOx capture processes. At the optimal blending ratio, the recycling efficiency of each seashell in the Republic of Korea was achieved from a minimum of 2.72 % to a maximum of 99.15 % annually. Therefore, this work makes a significant contribution to the chemical engineering field, providing an environmentally friendly, economical process and an efficient simultaneous optimization framework applied to the proposed model.

Cancer Surveillance in Solid Organ Transplant Recipients With a Pretransplant History of Malignancy: Multidisciplinary Collaborative Expert Opinion.

With improved medical treatments, the prognosis for many malignancies has improved, and more patients are presenting for transplant evaluation with a history of treated cancer. Solid organ transplant (SOT) recipients with a prior malignancy are at higher risk of posttransplant recurrence or de novo malignancy, and they may require a cancer surveillance program that is individualized to their specific needs. There is a dearth of literature on optimal surveillance strategies specific to SOT recipients. A working group of transplant physicians and cancer-specific specialists met to provide expert opinion recommendations on optimal cancer surveillance after transplantation for patients with a history of malignancy. Surveillance strategies provided are mainly based on general population recurrence risk data, immunosuppression effects, and limited transplant-specific data and should be considered expert opinion based on current knowledge. Prospective studies of cancer-specific surveillance models in SOT recipients should be supported to inform posttransplant management of this high-risk population.

Child face recognition at scale: synthetic data generation and performance benchmark

We address the need for a large-scale database of children’s faces by using generative adversarial networks (GANs) and face-age progression (FAP) models to synthesize a realistic dataset referred to as “HDA-SynChildFaces”. Hence, we proposed a processing pipeline that initially utilizes StyleGAN3 to sample adult subjects, which is subsequently progressed to children of varying ages using InterFaceGAN. Intra-subject variations, such as facial expression and pose, are created by further manipulating the subjects in their latent space. Additionally, this pipeline allows the even distribution of the races of subjects, allowing the generation of a balanced and fair dataset with respect to race distribution. The resulting HDA-SynChildFaces consists of 1,652 subjects and 188,328 images, each subject being present at various ages and with many different intra-subject variations. We then evaluated the performance of various facial recognition systems on the generated database and compared the results of adults and children at different ages. The study reveals that children consistently perform worse than adults on all tested systems and that the degradation in performance is proportional to age. Additionally, our study uncovers some biases in the recognition systems, with Asian and black subjects and females performing worse than white and Latino-Hispanic subjects and males.

AI-Driven Synthetization Pipeline of Realistic 3D-CT Data for Industrial Defect Segmentation

Training data is crucial for any artificial intelligence model. Previous research has shown that various methods can be used to enhance and improve AI training data. Taking a step beyond previous research, this paper presents a method that uses AI techniques to generate CT training data, especially realistic, artificial, industrial 3D voxel data. This includes that material as well as realistic internal defects, like pores, are artificially generated. To automate the processes, the creation of the data is implemented in a 3D Data Generation, called SPARC (Synthetized Process Artificial Realistic CT data). The SPARC is built as a pipeline consisting of several steps where different types of AI fulfill different tasks in the process of generating synthetic data. One AI generates geometrically realistic internal defects. Another AI is used to generate a realistic 3D voxel representation. This involves a conversion from STL to voxel data and generating the gray values accordingly. By combining the different AI methods, the SPARC pipeline can generate realistic 3D voxel data with internal defects, addressing the lack of data for various applications. The data generated by SPARC achieved a structural similarity of 98% compared to the real data. Realistic 3D voxel training data can thus be generated. For future AI applications, annotations of various features can be created to be used in both supervised and unsupervised training.

Exploring Innovative Approaches to Synthetic Tabular Data Generation

The rapid advancement of data generation techniques has spurred innovation across multiple domains. This comprehensive review delves into the realm of data generation methodologies, with a keen focus on statistical and machine learning-based approaches. Notably, novel strategies like the divide-and-conquer (DC) approach and cutting-edge models such as GANBLR have emerged to tackle a spectrum of challenges, spanning from preserving intricate data relationships to enhancing interpretability. Furthermore, the integration of generative adversarial networks (GANs) has sparked a revolution in data generation across sectors like healthcare, cybersecurity, and retail. This review meticulously examines how these techniques mitigate issues such as class imbalance, data scarcity, and privacy concerns. Through a meticulous analysis of evaluation metrics and diverse applications, it underscores the efficacy and potential of synthetic data in refining predictive models and decision-making software. Concluding with insights into prospective research trajectories and the evolving role of synthetic data in propelling machine learning and data-driven solutions across disciplines, this work provides a holistic understanding of the transformative power of contemporary data generation methodologies.

Wirtschaftszahlen der GJAE-Ausgabe 1/2024 (Stand: 15.02.2024)

Wirtschaftszahlen: This database contains general data on the development of agricultural producer and input prices, the consumer price index and foreign trade in agricultural and food goods on a monthly basis for the previous 24 months. In addition, this database comprises important volume and price developments on national and international markets for selected product groups of agricultural products such as grain, livestock, meat, milk and dairy products as well as vegetables, fruit and sugar. This file is only available in German.

Experimental Study on Using Synthetic Images as a Portion of Training Dataset for Object Recognition in Construction Site

The application of Artificial Intelligence (AI) across various industries necessitates the acquisition of relevant environmental data and the implementation of AI recognition learning based on this data. However, the data available in real-world environments are limited and difficult to obtain. Construction sites represent dynamic and hazardous environments with a significant workforce, making data acquisition challenging and labor-intensive. To address these issues, this experimental study explored the potential of generating synthetic data to overcome the challenges of obtaining data from hazardous construction sites. Additionally, this research investigated the feasibility of hybrid dataset in securing construction-site data by creating synthetic data for scaffolding, which has a high incidence of falls but low object recognition rates due to its linear object characteristics. We generated a dataset by superimposing scaffolding objects, from which the backgrounds were removed, onto various construction site background images. Using this dataset, we produced a hybrid dataset to assess the feasibility of synthetic data for construction sites and to evaluate improvements in object recognition performance. By finding the optimal composition ratio with real data and conducting model training, the highest accuracy was achieved at an 8:2 ratio, with a construction object recognition accuracy of 0.886. Therefore, this study aims to reduce the risk and labor associated with direct data collection at construction sites through a hybrid dataset, achieving data generation at a low cost and high efficiency. By generating synthetic data to find the optimal ratio and constructing a hybrid dataset, this research demonstrates the potential to address the problems of data scarcity and data quality on construction sites. The improvement in recognition accuracy of the construction safety management system is anticipated, suggesting that the creation of synthetic data for constructing a hybrid dataset can reduce construction safety-accident issues.

A generative benchmark for evaluating the performance of fluorescent cell image segmentation

Fluorescent cell imaging technology is fundamental in life science research, offering a rich source of image data crucial for understanding cell spatial positioning, differentiation, and decision-making mechanisms. As the volume of this data expands, precise image analysis becomes increasingly critical. Cell segmentation, a key analysis step, significantly influences quantitative analysis outcomes. However, selecting the most effective segmentation method is challenging, hindered by existing evaluation methods' inaccuracies, lack of graded evaluation, and narrow assessment scope. Addressing this, we developed a novel framework with two modules: StyleGAN2-based contour generation and Pix2PixHD-based image rendering, producing diverse, graded-density cell images. Using this dataset, we evaluated three leading cell segmentation methods: DeepCell, CellProfiler, and CellPose. Our comprehensive comparison revealed CellProfiler's superior accuracy in segmenting cytoplasm and nuclei. Our framework diversifies cell image data generation and systematically addresses evaluation challenges in cell segmentation technologies, establishing a solid foundation for advancing research and applications in cell image analysis.

TIAP: an observational procedure for assessing family relationships: a clinical case from the parenting evaluation context

TIAP is an observational procedure to assess family functioning detecting simultaneously the role of each participant and the interdependence of relational behaviors. In particular, the procedure requires family members to play according to different interactive configurations (parent1-children; parent2-children, all together, children and parents as separate units) and therefore different microtransitions from one configuration to another. As such, the procedure allows to study how family members coordinate to maintain stability, promote change, and encourage members to explore different interactive configurations within the family system. TIAP has been validated through several studies conducted with different non-clinical groups of families that have highlighted the salient aspects of family functioning, and significant correlations with variables external to the family system, such as children’s social–emotional competence in the educational context. This paper focuses on the use of TIAP in the contexts of assessing parental competence. Specifically, the article aims to describe, through the reference to a clinical case, the results emerged from a study conducted with 33 families involved in a parenting assessment process. The study is part of a broader collaborative project between the Child and Adolescent Neuropsychiatry Clinic of the Italian National Health Service in Parma, the University of Parma, and the Bologna Family Therapy Center. TIAP was administered to all the families involved as a complement to other tools routinely used for all cases handled by the professionals of the clinic. The coding system includes different indices. Some analyze the interactive family modes: family coordination (mutual attention and responsiveness), the responses to potentials for change (disregard, absorption, amplification), and intra-familiar exploration. Other indices concern the quality of the interactions: the relational triadic dynamic of microtransition (detaching-entrusting-welcoming-joining) and the consistency/inconsistency of the communication channels. The results highlighted how TIAP makes it possible to identify the specific interactive modalities of the different members and their interdependence and reciprocity, favoring the identification of both family weaknesses and family resources, including the children’s contribution. Furthermore, the general data trend showed that TIAP indices detect some important prognostic elements capable of guiding the court’s decisions.

Life, death, and statins: association of statin prescriptions and survival in older general practice patients.

This study serves as an exemplar to demonstrate the scalability of a research approach using survival analysis applied to general practice electronic health record data from multiple sites. Collection of these data, the subsequent analysis, and the preparation of practice-specific reports were performed using a bespoke distributed data collection and analysis software tool. Statins are a very commonly prescribed medication, yet there is a paucity of evidence for their benefits in older patients. We examine the relationship between statin prescriptions for general practice patients over 75 and all-cause mortality. We carried out a retrospective cohort study using survival analysis applied to data extracted from the electronic health records of five Australian general practices. The data from 8025 patients were analysed. The median duration of follow-up was 6.48 years. Overall, 52015 patient-years of data were examined, and the outcome of death from any cause was measured in 1657 patients (21%), with the remainder being censored. Adjusted all-cause mortality was similar for participants not prescribed statins versus those who were (HR 1.05, 95% CI 0.92-1.20, P = 0.46), except for patients with diabetes for whom all-cause mortality was increased (HR = 1.29, 95% CI: 1.00-1.68, P = 0.05). In contrast, adjusted all-cause mortality was significantly lower for patients deprescribed statins compared to those who were prescribed statins (HR 0.81, 95% CI 0.70-0.93, P < 0.001), including among females (HR = 0.75, 95% CI: 0.61-0.91, P < 0.001) and participants treated for secondary prevention (HR = 0.72, 95% CI: 0.60-0.86, P < 0.001). This study demonstrated the scalability of a research approach using survival analysis applied to general practice electronic health record data from multiple sites. We found no evidence of increased mortality due to statin-deprescribing decisions in primary care.

A template wizard for the cocreation of machine-readable data-reporting to harmonize the evaluation of (nano)materials.

Making research data findable, accessible, interoperable and reusable (FAIR) is typically hampered by a lack of skills in technical aspects of data management by data generators and a lack of resources. We developed a Template Wizard for researchers to easily create templates suitable for consistently capturing data and metadata from their experiments. The templates are easy to use and enable the compilation of machine-readable metadata to accompany data generation and align them to existing community standards and databases, such as eNanoMapper, streamlining the adoption of the FAIR principles. These templates are citable objects and are available as online tools. The Template Wizard is designed to be user friendly and facilitates using and reusing existing templates for new projects or project extensions. The wizard is accompanied by an online template validator, which allows self-evaluation of the template (to ensure mapping to the data schema and machine readability of the captured data) and transformation by an open-source parser into machine-readable formats, compliant with the FAIR principles. The templates are based on extensive collective experience in nanosafety data collection and include over 60 harmonized data entry templates for physicochemical characterization and hazard assessment (cell viability, genotoxicity, environmental organism dose-response tests, omics), as well as exposure and release studies. The templates are generalizable across fields and have already been extended and adapted for microplastics and advanced materials research. The harmonized templates improve the reliability of interlaboratory comparisons, data reuse and meta-analyses and can facilitate the safety evaluation and regulation process for (nano) materials.

Relationship Between the Gut Microbiota and Neurological Deficits in Patients With Cerebral Ischemic Stroke.

The aim of the paper was to investigate the composition and structure of intestinal flora in patients with cerebral ischemic stroke (CIS), and to investigate the relationship between gut microbiota (GM) and different levels of stroke severity. In this study, 47 CIS patients (16 mild, 21 moderate, and 10 severe) and 15 healthy controls were included. General information, clinical data, and behavioral scores of the enrolled subjects were collected. Deoxyribonucleic acid in fecal intestinal flora was extracted and detected using high-throughput Illumina 16S ribosomal ribonucleic acid sequencing technology. Finally, the correlation between the community composition of intestinal microbiota and National Institutes of Health Stroke Scale (NIHSS) score in CIS patients was analyzed. Compared with healthy controls, there was no statistically significant difference in Alpha diversity among CIS patients, but the principal coordinate analysis showed significant differences in the composition of the GM among stroke patients with different degrees of severity and controls. In CIS patients, Streptococcus was significantly enriched, and Eshibacter-Shigella, Bacteroides, and Agathobacter were significantly down-regulated (P < .05). In addition, the relative abundance of Blautia was negatively correlated with the NIHSS score. Our results show that different degrees of CIS severity exert distinct effects on the intestinal microbiome. This study reveals the intestinal microecological changes after brain injury from the perspective of brain-gut axis. Intestinal microorganisms not only reveal the possible pathological process and indicate the severity of neurologic impairment, but also make targeted therapy possible for CIS patients.

Filling the Void: How E.U. Privacy Law Spills Over to the U.S.

There is mounting evidence that the EU’s General Data Protection Regulation (GDPR) has influenced the information privacy policies and practices that firms adopt in relation to people outside of the E.U., even when that is not required by the E.U. regulation. We use a hand-coded dataset of privacy policies from firms’ U.S. and E.U.-facing websites to document and explain these kinds of international regulatory spillovers. Our findings are consistent with the hypothesis that spillovers are driven by the costs of complying with different standards in different parts of the same firm. In fact, 75% of the firms in our sample use the same privacy policy for their U.S. and E.U.-facing websites. At the same time, our findings do not support the conclusion that firms comply with the GDPR in their U.S.-facing privacy policies out of fear of being sanctioned if the policy is somehow applied to E.U. residents. Finally, we find that spillovers are more prevalent among firms with a physical presence in the E.U. This suggests that international networks of compliance professionals may play a significant and understudied role in regulatory compliance, perhaps by providing channels for norms and resources to move across borders.

OHDSI-compliance: a set of document templates facilitating the implementation and operation of a software stack for real-world evidence generation

IntroductionThe open-source software offered by the Observational Health Data Science and Informatics (OHDSI) collective, including the OMOP-CDM, serves as a major backbone for many real-world evidence networks and distributed health data analytics platforms. While container technology has significantly simplified deployments from a technical perspective, regulatory compliance can remain a major hurdle for the setup and operation of such platforms. In this paper, we present OHDSI-Compliance, a comprehensive set of document templates designed to streamline the data protection and information security-related documentation and coordination efforts required to establish OHDSI installations.MethodsTo decide on a set of relevant document templates, we first analyzed the legal requirements and associated guidelines with a focus on the General Data Protection Regulation (GDPR). Moreover, we analyzed the software architecture of a typical OHDSI stack and related its components to the different general types of concepts and documentation identified. Then, we created those documents for a prototypical OHDSI installation, based on the so-called Broadsea package, following relevant guidelines from Germany. Finally, we generalized the documents by introducing placeholders and options at places where individual institution-specific content will be needed.ResultsWe present four documents: (1) a record of processing activities, (2) an information security concept, (3) an authorization concept, as well as (4) an operational concept covering the technical details of maintaining the stack. The documents are publicly available under a permissive license.DiscussionTo the best of our knowledge, there are no other publicly available sets of documents designed to simplify the compliance process for OHDSI deployments. While our documents provide a comprehensive starting point, local specifics need to be added, and, due to the heterogeneity of legal requirements in different countries, further adoptions might be necessary.