Adaptive Data Research Articles

Data-driven program adaptation and continuous improvement: A case study of the Adolescents 360 (A360) project in Ethiopia, Nigeria, and Tanzania

Interventions benefit from ongoing continuous quality improvement during implementation to refine and strengthen them in response to changing contexts and emerging opportunities. This article reflects on the experience from the Adolescents 360 (A360) project in implementing user-centered, adaptive programming across Tanzania, Ethiopia, and Nigeria from 2018 to 2020. During this implementation period, A360 progressed through three distinct phases: an ‘optimization’ of project interventions in pursuit of efficiency and preparation for scale; a re-focusing on user-centered programming to go beyond efficiency and pursue greater impact; and a rapid course correction in response to the emerging COVID-19 pandemic. A360’s approach to data-informed adaptation evolved throughout these phases. From a hyper focus on cost-efficiency during optimization to a more expansive use of mixed methods data for adaptation during its second phase and finally iteration in response to an extremely restrictive service delivery environment during the COVID-19 pandemic. In this article we reflect on how these periods of adaptation and course correction contributed to the evolution of the project’s interventions and provided significant lessons which the project has taken into its new investment phase. Balance is key – utilizing a range of mixed methods data instead of a singular focus on cost-efficiency, streamlining interventions while not compromising long-term impact, and retaining a user-focus even while considering sustainability.

A Small Database with an Adaptive Data Selection Method for Solder Joint Fatigue Life Prediction in Advanced Packaging.

There has always been high interest in predicting the solder joint fatigue life in advanced packaging with high accuracy and efficiency. Artificial Intelligence Plus (AI+) is becoming increasingly popular as computational facilities continue to develop. This study will introduce machine learning (a core component of AI). With machine learning, metamodels that approximate the attributes of systems or functions are created to predict the fatigue life of advanced packaging. However, the prediction ability is highly dependent on the size and distribution of the training data. Increasing the amount of training data is the most intuitive approach to improve prediction performance, but this implies a higher computational cost. In this research, the adaptive sampling methods are applied to build the machine learning model with a small dataset sampled from an existing database. The performance of the model will be visualized using predefined criteria. Moreover, ensemble learning can be used to improve the performance of AI models after they have been fully trained.

ADAC: an active domain adaptive network with progressive learning strategy for cloud detection of remote sensing imagery

ABSTRACT To enhance the adaptability and application capability of the cloud detection model in different remote sensing satellite domains, unsupervised domain adaptation methods are employed to improve the model’s robustness and transferability. However, conducting fully unlabeled training in the target domain to be adapted may impose difficulty on further improving the accuracy. It is more practical to obtain a few labeled target data for domain adaptation. Additionally, the detection of complex cloud scenes remains a persistent challenge. To address these issues, an Active Domain Adaptation network for Cloud detection (ADAC) is proposed in this paper. This network employs active learning to select a small number of representative samples for annotation, facilitating the transfer of a source domain-trained model to the target domain. Moreover, a novel Generator (G) designed for cloud segmentation is developed, where G utilizes a progressive learning strategy by dividing the network into two stages to learn gradient information, guiding cloud boundary segmentation and enabling the distinction between cloud-like objects and clouds. Furthermore, G integrates Transformer and Convolutional Neural Network to address the issues of misclassifying thin clouds. The proposed method is validated on Landsat 8 and GF-2 datasets, representing the source and target domains, respectively. The experimental results demonstrate ADAC achieved a 4% improvement in the overall accuracy with only 5% labeled training samples in the target domain, achieving the effective adaptation performance in the GF-2 domain.

Adaptive data quality scoring operations framework using drift-aware mechanism for industrial applications

Within data-driven artificial intelligence (AI) systems for industrial applications, ensuring the reliability of the incoming data streams is an integral part of trustworthy decision-making. An approach to assess data validity is data quality scoring, which assigns a score to each data point or stream based on various quality dimensions. However, certain dimensions exhibit dynamic qualities, which require adaptation on the basis of the system’s current conditions. Existing methods often overlook this aspect, making them inefficient in dynamic production environments. In this paper, we introduce the Adaptive Data Quality Scoring Operations Framework, a novel framework developed to address the challenges posed by dynamic quality dimensions in industrial data streams. The framework introduces an innovative approach by integrating a dynamic change detector mechanism that actively monitors and adapts to changes in data quality, ensuring the relevance of quality scores. We evaluate the proposed framework performance in a real-world industrial use case. The experimental results reveal high predictive performance and efficient processing time, highlighting its effectiveness in practical quality-driven AI applications.

Multi-strategy adaptive data augmentation for Graph Neural Networks

While existing Graph Neural Networks (GNNs) have demonstrated exceptional performance in semi-supervised learning, they are criticized for not fully utilizing unlabeled data and graph structure, leading to over-fitting. Recently, data augmentation as a technique gaining significant attention recently for enhancing the generalization capabilities of Graph Neural Networks (GNNs), is regarded as a promising solution to mitigate the issue of over-fitting. Nonetheless, existing studies focus on designing a simple augmentation strategy tailored to a specific dataset, without ensuring a diverse and consistent distribution of augmented graphs during the training process. Moreover, formulating different graph augmentation strategies for diverse datasets relies heavily on domain knowledge due to the distinct distributions inherent in various graphs. Designing a universal augmentation strategy which can adapt to different datasets remains a challenging endeavor. To resolve the thorny issue, we introduce an adaptive augmentation framework called MSA-AUG, which can adaptively integrate multiple heuristic strategies for GNNs. Specifically, in order to enhance the representation quality of GNNs, we first take advantage of various augmentation strategies from global, local and label perspectives. Then we integrate them into a set of candidate strategies, which is model-agnostic for any GNNs. Then, we construct a search space of candidate strategies and take each candidate-strategy with different selection weights and operation magnitudes. The learned parameters are leveraged to explore different selection weights and operation magnitudes for ensuring that the contribution of various graph augmentation strategies is well-adjusted. Finally, we employ a search algorithm based on density matching to dynamically explore the candidate strategies and take the best augmentation strategy on various datasets with different distributions for improving the generalization ability of GNNs. Extensive experiments on diverse benchmark datasets demonstrate that MSA-AUG can improve the performance of GNNs with varying structures of backbones.

Adaptive Imputation of Irregular Truncated Signals with Machine Learning

In modern advanced manufacturing systems, the use of smart sensors and other Internet of Things (IoT) technology to provide real-time feedback to operators about the condition of various machinery or other equipment is prevalent. A notable issue in such IoT-based advanced manufacturing systems is the problem of connectivity, where a dropped Internet connection can lead to the loss of important condition data from a machine. Such gaps in the data, which we call irregular truncated signals, can lead to incorrect assumptions about the status of a machine and other flawed decision-making processes. This paper presents an adaptive data imputation framework based on machine learning (ML) algorithms to assess whether the missing data in a signal is missing completely at random (MCAR), missing at random (MAR), or missing not at random (MNAR) and automatically select an appropriate ML-based data imputation model to deal with the missing data. Our results demonstrate the potential for applying ML algorithms to the challenge of irregularly truncated signals, as well as the capability of our adaptive framework to intelligently solve this issue.

Multichannel Adaptive Data Mixture Augmentation for Graph Neural Networks

Graph neural networks (GNNs) have demonstrated significant potential in analyzing complex graph-structured data. However, conventional GNNs encounter challenges in effectively incorporating global and local features. Therefore, this paper introduces a novel approach for GNN called multichannel adaptive data mixture augmentation (MAME-GNN). It enhances a GNN by adopting a multi-channel architecture and interactive learning to effectively capture and coordinate the interrelationships between local and global graph structures. Additionally, this paper introduces the polynomial–Gaussian mixture graph interpolation method to address the problem of single and sparse graph data, which generates diverse and nonlinear transformed samples, improving the model's generalization ability. The proposed MAME-GNN is validated through extensive experiments on publicly available datasets, showcasing its effectiveness. Compared to existing GNN models, the MAME-GNN exhibits superior performance, significantly enhancing the model's robustness and generalization ability.

Integrating agile methodologies into data governance: Achieving flexibility and control simultaneously

This review paper explores integrating agile methodologies into data governance to achieve flexibility and control. Agile methodologies, known for their iterative development, collaboration, and responsiveness to change, can enhance the adaptability of data governance frameworks. This paper analyzes the alignment of agile principles with data governance, discusses strategies for integration, such as adaptive data governance and incremental implementation, and examines how to balance flexibility with the necessary controls for effective data governance. Key practices, challenges, and potential solutions focus on fostering a collaborative culture and leveraging appropriate tools and technologies. The paper also suggests future research directions, emphasizing the role of emerging technologies and evolving regulatory requirements. Ultimately, this integration enhances decision-making, operational efficiency, and organizational competitive advantage.

Towards Collaborative Edge Intelligence: Blockchain-Based Data Valuation and Scheduling for Improved Quality of Service

Collaborative edge intelligence, a distributed computing paradigm, refers to a system where multiple edge devices work together to process data and perform distributed machine learning (DML) tasks locally. Decentralized Internet of Things (IoT) devices share knowledge and resources to improve the quality of service (QoS) of the system with reduced reliance on centralized cloud infrastructure. However, the paradigm is vulnerable to free-riding attacks, where some devices benefit from the collective intelligence without contributing their fair share, potentially disincentivizing collaboration and undermining the system’s effectiveness. Moreover, data collected from heterogeneous IoT devices may contain biased information that decreases the prediction accuracy of DML models. To address these challenges, we propose a novel incentive mechanism that relies on time-dependent blockchain records and multi-access edge computing (MEC). We formulate the QoS problem as an unbounded multiple knapsack problem at the network edge. Furthermore, a decentralized valuation protocol is introduced atop blockchain to incentivize contributors and disincentivize free-riders. To improve model prediction accuracy within latency requirements, a data scheduling algorithm is given based on a curriculum learning framework. Based on our computer simulations using heterogeneous datasets, we identify two critical factors for enhancing the QoS in collaborative edge intelligence systems: (1) mitigating the impact of information loss and free-riders via decentralized data valuation and (2) optimizing the marginal utility of individual data samples by adaptive data scheduling.

[formula omitted] : An adaptive data aggregation algorithm for IoT infrastructure

In IoT infrastructure, high-frequency sensing and subsequent transmission of sensed data to computational facilities can lead to redundant data storage and processing, consuming significant storage and processing capacity. As a result, the IoT infrastructure needs more data transmission cycles, leading to data redundancy and low network up-time due to the drainage of limited battery capacity. Conversely, if the data is communicated at a lower rate, it may cause absolute data delivery to the processing unit, which is useless. As a result, a well-designed data aggregation algorithm is required. This paper proposes the ADA2−IoT, an Adaptive Data Aggregation Algorithm for IoT Infrastructure tailored to optimize parameters such as low data redundancy, limited data communication cycles, and high IoT infrastructure up times. The proposed algorithm consists of two key components: the Route Data Aggregator (RDA) performs aggregation during data transit towards the Edge node or gateway, and the Node Data Aggregator (NDA) performs data aggregation during capturing or sensing data. The algorithm employs metrics like Age of Information (AoI) and data freshness factor during the node and route data aggregation phase to capture and timely deliver data to the Edge node, where this data is processed for informed decision-making. The proposed algorithm was efficiently tested on a simulation and IoT hardware deployment environment. Both simulation and hardware results demonstrate a substantial improvement in QoS parameters, such as a decrease in data redundancy and packet exchanges, leading to considerable energy savings and prolonging the lifespan of IoT infrastructure.

Rapid Online Data Collection and Reporting to Local Health Departments during the Pandemic: The COVID-19 Outbreak Public Evaluation (COPE) Initiative SARS-CoV-2 Case Surveys.

To develop and implement a pilot online data collection tool to help local health departments with their COVID-19 pandemic response efforts and inform health department actions. The COVID-19 Outbreak Public Evaluation (COPE) was an online survey and was distributed by participating sites to individuals who recently tested positive for SARS-CoV-2. Surveys recorded participant demographics and assessed recent infection risk behaviors (eg, mask use, air travel), vaccination status, sleep and exercise habits, social behaviors and beliefs, and physical and mental health. Seven health departments participated in the initiative, which took place during May 1 to September 30, 2022. Identical items were administered to demographically representative samples of adults nationally in the United States within a similar timeframe. A total of 38555 participants completed surveys. Responses from participants with recent SARS-CoV-2 infections were compared with respondents from the national surveys who did not have evidence or awareness of prior SARS-CoV-2 infections. To implement of a process that allows health departments to receive data from local cases and compare this information to national controls during the COVID-19 pandemic. Fifty-four biweekly reports were provided to public health departments between May and September 2022. Information and comparisons within the reports were updated in response to evolving public health priorities for the pandemic response. The initiative helped to guide public health response efforts during the COVID-19 pandemic. Moreover, the receptiveness by local health departments and participants provides evidence to support this data collection and reporting model as a component of the public health response to future emergencies. This project demonstrates the feasibility of a centralized, rapid, and adaptive data collection system for local health departments and provides evidence to advocate for data collection methods to help guide local health departments to respond in a timely and effective manner to future public health emergencies.

Improving rigor and reproducibility in chromatin immunoprecipitation assay data analysis workflows with Rocketchip.

As genome sequencing technologies advance, the accumulation of sequencing data in public databases necessitates more robust and adaptable data analysis workflows. Here, we present Rocketchip, which aims to offer a solution to this problem by allowing researchers to easily compare and swap out different components of ChIP-seq, CUT&RUN, and CUT&Tag data analysis, thereby facilitating the identification of reliable analysis methodologies. Rocketchip enables researchers to efficiently process large datasets while ensuring reproducibility and allowing for the reanalysis of existing data. By supporting comparative analyses across different datasets and methodologies, Rocketchip contributes to the rigor and reproducibility of scientific findings. Furthermore, Rocketchip serves as a platform for benchmarking algorithms, allowing researchers to identify the most accurate and efficient analytical approaches to be applied to their data. In emphasizing reproducibility and adaptability, Rocketchip represents a significant step towards fostering robust scientific research practices.

'There was nothing, just absolute darkness': Understanding the needs of those caring for children and young people with complex neurodisability in a diverse UK context: A qualitative exploration in the ENCOMPASS study.

Children and young people (CYP) with complex neurodisability experience multiple physical, communication, educational and social challenges, which require complex packages of multidisciplinary care. Part of the holistic care required includes supporting the families and parents/caregivers. The aim of the wider study was to introduce a new programme ('Ubuntu') to parents/caregivers and healthcare professionals (HCPs) in order to test the feasibility and acceptability of the concept and content, with the goal of potential adaptation for the UK in mind. Data collection and analysis uncovered rich data on caregiving journeys, navigation of health services, and perceived service gaps. This paper focuses solely on these topics. Further papers will report on the feasibility and adaptation data. Two rounds of semi-structured interviews were conducted with 12 caregivers of CYP with complex neurodisability and six HCPs from a variety of disciplines, recruited from a community child health service in London Borough of Newham, UK in 2020. The interviews included open-ended questions to explore caregiving journeys, experiences of navigating health services and perceived service gaps. Transcripts were analysed using a data-driven inductive thematic analysis. Three themes were identified that related to the aim of understanding caregivers' experiences and unmet needs relating to current service provision. These were (1) Caregiver Mental Health, (2) The Information Gap and (3) The Need for Holistic Support. Mental health difficulties were reported, particularly around the period of diagnosis. Priority needs included the provision of clear information about the diagnosis and services offered, opportunities to forge peer support networks and for services across the community to collaborate. The delivery of health services for CYP with neurodisability should encompass the broad needs of the family as well as meeting the clinical needs of the CYP.

Adaptive Graph-Based Support Vector Data Description for Weakly-Supervised Anomaly Detection

Adaptive Graph-Based Support Vector Data Description for Weakly-Supervised Anomaly Detection

Extended nnU-Net for Brain Metastasis Detection and Segmentation in Contrast-Enhanced Magnetic Resonance Imaging With a Large Multi-Institutional Data Set

PurposeTo investigate an extended self-adapting nnU-Net framework for detecting and segmenting brain metastases (BM) on MRI. ApproachSix different nnU-Net systems with adaptive data sampling, adaptive Dice loss (ADL) or different patch/batch sizes were trained and tested for detecting and segmenting intraparenchymal BM with a size ≥ 2 mm on 3D post-Gd T1-weighted MRI volumes using 2092 patients from seven institutions (1712, 195 and 185 patients for training, validation and testing, respectively). Gross tumor volumes (GTVs) of BM delineated by physicians for stereotactic radiosurgery (SRS) were collected retrospectively and curated at each institute. Additional centralized data curation was carried out to create GTVs of uncontoured BM by two radiologists to improve accuracy of ground truth. The training dataset was augmented with synthetic BMs of 1025 MRI volumes using a 3D generative pipeline. BM detection was evaluated by lesion-level sensitivity and false-positive (FP) rate. BM segmentation was assessed by lesion-level Dice similarity coefficient (DSC), 95-percentile Hausdorff distance (HD95) and average HD. The performances were assessed across different BM sizes. Additional testing was performed using a second dataset of 206 patients. ResultsOf the six nnU-Net systems, the nnU-Net with ADL achieved the best detection and segmentation performance on the first testing dataset. At an FP rate of 0.65±1.17, overall sensitivity was 0.904 for all sizes of BM, 0.966 for BM ≥ 0.1 cm3 and 0.824 for BM < 0.1 cm3. Mean values of DSC, HD95 and average HD of all detected BMs were 0.758, 1.45 mm and 0.23 mm, respectively. Performances on the second testing dataset achieved sensitivity of 0.907 at an FP rate of 0.57±0.85 for all BM sizes, and average HD of 0.33 mm for all detected BM. ConclusionsOur proposed extension of the self-configuring nnU-Net framework substantially improved small BM detection sensitivity while maintaining a controlled FP rate. Clinical utility of the extended nnU-Net model for assisting early BM detection and SRS planning will be investigated.

AdaptChain: Adaptive Data Sharing and Synchronization for NFV Systems on Heterogeneous Architectures

AdaptChain: Adaptive Data Sharing and Synchronization for NFV Systems on Heterogeneous Architectures

Precise Adverse Weather Characterization by Deep-Learning-Based Noise Processing in Automotive LiDAR Sensors

With current advances in automated driving, optical sensors like cameras and LiDARs are playing an increasingly important role in modern driver assistance systems. However, these sensors face challenges from adverse weather effects like fog and precipitation, which significantly degrade the sensor performance due to scattering effects in its optical path. Consequently, major efforts are being made to understand, model, and mitigate these effects. In this work, the reverse research question is investigated, demonstrating that these measurement effects can be exploited to predict occurring weather conditions by using state-of-the-art deep learning mechanisms. In order to do so, a variety of models have been developed and trained on a recorded multiseason dataset and benchmarked with respect to performance, model size, and required computational resources, showing that especially modern vision transformers achieve remarkable results in distinguishing up to 15 precipitation classes with an accuracy of 84.41% and predicting the corresponding precipitation rate with a mean absolute error of less than 0.47 mm/h, solely based on measurement noise. Therefore, this research may contribute to a cost-effective solution for characterizing precipitation with a commercial Flash LiDAR sensor, which can be implemented as a lightweight vehicle software feature to issue advanced driver warnings, adapt driving dynamics, or serve as a data quality measure for adaptive data preprocessing and fusion.

Unsupervised Graph Structure Learning Based on Optimal Graph Topology Modeling and Adaptive Data Augmentation

Unsupervised Graph Structure Learning Based on Optimal Graph Topology Modeling and Adaptive Data Augmentation

Could this be next for corpus linguistics? Methods of semi-automatic data annotation with contextualized word embeddings

Abstract This paper explores how linguistic data annotation can be made (semi-)automatic by means of machine learning. More specifically, we focus on the use of “contextualized word embeddings” (i.e. vectorized representations of the meaning of word tokens based on the sentential context in which they appear) extracted by large language models (LLMs). In three example case studies, we assess how the contextualized embeddings generated by LLMs can be combined with different machine learning approaches to serve as a flexible, adaptable semi-automated data annotation tool for corpus linguists. Subsequently, to evaluate which approach is most reliable across the different case studies, we use a Bayesian framework for model comparison, which estimates the probability that the performance of a given classification approach is stronger than that of an alternative approach. Our results indicate that combining contextualized word embeddings with metric fine-tuning yield highly accurate automatic annotations.

Analysis of Emotional Tendency of Tourism Consumers Based on LSTM and Fuzzy Control Algorithm

Learning the emotional tendency of travelers improves their interests and provide optimal traveling recommendations. This, however, requires large volumes of data such as travel plans, visit sites, personal interests, value for money, etc. for a detailed analysis. For ease of such analysis, in this article, an organized combinational control method (OC2M) is proposed. This method relies on conventional long short-term memory (LSTM) and fuzzy control (FC) to support such analysis. The first is responsible for filtering non-repeated data from the previous travel/ tour history reducing the data discreteness. This is organized non-recurrently to prevent outdated/ trivial data from influencing the consumer’s emotional learning. The FC process filters the adaptable data with the future tour/ travel plan for providing optimal recommendations that are liable to the consumer’s emotional tendency. This is identified based on the user's interest, preferences, and emotional connectivity with the place or plan from previous histories. Therefore, the proposed method improves the recommendation and validation tendencies of the consumer’s travel plan accordingly.