Data Distribution Research Articles

Causality-inspired crop pest recognition based on Decoupled Feature Learning.

Ensuring the efficient recognition and management of crop pests is crucial for maintaining the balance in global agricultural ecosystems and ecological harmony. Deep learning-based methods have shown promise in crop pest recognition. However, prevailing methods often fail to address a critical issue: biased pest training dataset distribution stemming from the tendency to collect images primarily in certain environmental contexts, such as paddy fields. This oversight hampers recognition accuracy when encountering pest images dissimilar to training samples, highlighting the need for a novel approach to overcome this limitation. We introduce the Decoupled Feature Learning (DFL) framework, leveraging causal inference techniques to handle training dataset bias. DFL manipulates the training data based on classification confidence to construct different training domains and employs center triplet loss for learning class-core features. The proposed DFL framework significantly boosts existing baseline models, attaining unprecedented recognition accuracies of 95.33%, 92.59%, and 74.86% on the Li, DFSPD, and IP102 datasets, respectively. Extensive testing on three pest datasets using standard baseline models demonstrates the superiority of DFL in pest recognition. The visualization results show that DFL encourages the baseline models to capture the class-core features. The proposed DFL marks a pivotal step in mitigating the issue of data distribution bias, enhancing the reliability of deep learning in agriculture. © 2024 Society of Chemical Industry.

Structure-aware diffusion for low-dose CT imaging

Reducing the radiation dose leads to the x-ray computed tomography (CT) images suffering from heavy noise and artifacts, which inevitably interferes with the subsequent clinic diagnostic and analysis. Leading works have explored diffusion models for low-dose CT imaging to avoid the structure degeneration and blurring effects of previous deep denoising models. However, most of them always begin their generative processes with Gaussian noise, which has little or no structure priors of the clean data distribution, thereby leading to long-time inference and unpleasant reconstruction quality. To alleviate these problems, this paper presents a Structure-Aware Diffusion model (SAD), an end-to-end self-guided learning framework for high-fidelity CT image reconstruction. First, SAD builds a nonlinear diffusion bridge between clean and degraded data distributions, which could directly learn the implicit physical degradation prior from observed measurements. Second, SAD integrates the prompt learning mechanism and implicit neural representation into the diffusion process, where rich and diverse structure representations extracted by degraded inputs are exploited as prompts, which provides global and local structure priors, to guide CT image reconstruction. Finally, we devise an efficient self-guided diffusion architecture using an iterative updated strategy, which further refines structural prompts during each generative step to drive finer image reconstruction. Extensive experiments on AAPM-Mayo and LoDoPaB-CT datasets demonstrate that our SAD could achieve superior performance in terms of noise removal, structure preservation, and blind-dose generalization, with few generative steps, even one step only.

Enhanced Domain Adaptation for Foot Ulcer Segmentation Through Mixing Self-Trained Weak Labels.

Wound management requires the measurement of the wound parameters such as its shape and area. However, computerized analysis of the wound suffers the challenge of inexact segmentation of the wound images due to limited or inaccurate labels. It is a common scenario that the source domain provides an abundance of labeled data, while the target domain provides only limited labels. To overcome this, we propose a novel approach that combines self-training learning and mixup augmentation. The neural network is trained on the source domain to generate weak labels on the target domain via the self-training process. In the second stage, generated labels are mixed up with labels from the source domain to retrain the neural network and enhance generalization across diverse datasets. The efficacy of our approach was evaluated using the DFUC 2022, FUSeg, and RMIT datasets, demonstrating substantial improvements in segmentation accuracy and robustness across different data distributions. Specifically, in single-domain experiments, segmentation on the DFUC 2022 dataset scored a dice score of 0.711, while the score on the FUSeg dataset achieved 0.859. For domain adaptation, when these datasets were used as target datasets, the dice scores were 0.714 for DFUC 2022 and 0.561 for FUSeg.

Optimising synthetic datasets for machine learning-based prediction of building damage due to tunnelling

Assessment of tunnelling-induced building damage is a complex Soil-Structure Interaction (SSI) probelm, influenced by numerous geometric and material parameters of both the soil and structures, and is characterised by strong non-linear behaviour. Currently, there is a trend towards developing data-driven models using Machine Learning (ML) to capture this complex behaviour. Given the scarcity of real data, which typically comes from specific case studies, many researchers have turned to creating extensive synthetic datasets via sophisticated and validated numerical models like Finite Element Method (FEM). However, the development of these datasets and the training of advanced ML algorithms present significant challenges. poses significant challenges. Reliance solely on parameter domains and ranges derived from case studies can lead to imbalanced data distributions and subsequently poor performance of models in less populated regions. In this paper, we introduce a strategy for designing optimal high-confidence datasets through an iterative procedure. This process begins with a systematic literature review to determine the importance of parameters, their ranges, and dependencies as they pertain to building damage induced by SSI. Starting with several hundred FEM simulations, we generate an initial dataset and assess its quality and impact through Sensitivity Analysis (SA) studies, statistical modelling, and re-sampling in statistically significant regions. This evaluation allows us to refine the model’s input space, seeking scenarios that mitigate output distribution imbalances. The procedure is repeated until the datasets achieve a satisfactory balance for training metamodels, minimising bias effectively. Our findings highlight the success of this approach in identifying an optimal and feasible input space that significantly reduces imbalanced distributions of output features. This approach not only proves effective in our study but also offers a versatile methodology that could be adapted to other disciplines aiming to generate high-quality synthetic datasets.

Analysis of neural networks in text sentiment

The proliferation of digital media and online platforms has led to a significant increase in the production and distribution of textual data. Consequently, the study of text sentiment analysis has emerged as a crucial area of research within the domain of natural language processing. The objective of sentiment analysis is to detect and categorize emotional inclinations in text, which has substantial practical importance in comprehending public sentiment, monitoring brand reputation, and other related areas. This research does a comprehensive investigation of the utilization of neural network models in text sentiment analysis using literature review methods. It examines the difficulties encountered and the approaches employed to overcome these difficulties. The research encompasses the fundamental principles and techniques of text sentiment analysis utilizing neural networks, as well as the utilization of convolutional neural networks (CNN), recurrent neural networks (RNN)/long short-term memory networks (LSTM), attention mechanisms, and pre-trained models in the field of sentiment analysis. Furthermore, this study delves into the complexities associated with data noise, the interpretability of models, the analysis of long text and multimodal sentiment, as well as the utilization of transfer learning and domain adaptation techniques. The study's importance is in establishing a theoretical basis and practical recommendations for the advancement of sentiment analysis technology, particularly in enhancing model precision, comprehensibility, and range of applications, thereby delivering useful insights.

Wasserstein barycenter regression: application to the joint dynamics of regional GDP and life expectancy in Italy

AbstractWe propose to investigate the joint dynamics of regional gross domestic product and life expectancy in Italy through Wasserstein barycenter regression derived from optimal transport theory. Wasserstein barycenter regression has the advantage of being flexible in modeling complex data distributions, given its ability to capture multimodal relationships, while maintaining the possibility of incorporating uncertainty and priors, other than yielding interpretable results. The main findings reveal that regional clusters tend to emerge, highlighting inequalities in Italian regions in economic and life expectancy terms. This suggests that targeted policy actions at a regional level fostering equitable development, especially from an economic viewpoint, might reduce regional inequality. Our results are validated by a robustness check on a human mobility dataset and by an illustrative forecasting exercise, which confirms the model’s ability to estimate and predict joint distributions and produce novel empirical evidence.

Physics-Informed Neural Network for Turbulent Flow Reconstruction in Composite Porous-Fluid Systems

Abstract This study explores the implementation of Physics-Informed Neural Networks (PINN) to analyze turbulent flow in composite porous-fluid systems. These systems are composed of a fluid-saturated porous medium and an adjacent fluid, where the flow properties are exchanged across the porous-fluid interface. The PINN model employs a novel approach combining supervised learning and enforces fidelity to flow physics through penalization by the Reynolds-Averaged Navier-Stokes (RANS) equations. Two cases were simulated for this purpose: solid block, i.e., porous media with zero porosity, and porous block with a defined porosity. The effect of providing internal training data on the accuracy of the PINN predictions for prominent flow features including leakage, channeling effect and wake recirculation were investigated. Additionally, L2 norm error, which evaluates the prediction accuracy for flow variables was studied. Furthermore, PINN training time in both cases with internal training data were considered in this study. The results showed that the PINN predictions achieved high accuracy for the prominent flow features compared to the reference RANS data. In addition, second-order internal training data in the wall-normal direction reduced the L2 norm error by 100% for the solid block case, while for the porous block case, providing training data at the porous-fluid interface, increased the prediction accuracy by nearly 40% for second-order statistics. The study elucidates the impact of the internal training data distribution on the PINN training in complex turbulent flow dynamics, underscoring the necessity of turbulent second-order statistics variables in PINN training and an additional velocity gradient treatment to enhance PINN prediction.

Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet

Deep neural networks have played a crucial role in the field of deep learning, achieving significant success in practical applications. The architecture of neural networks is key to their performance. In the past few years, these architectures have been manually designed by experts with rich domain knowledge. Additionally, the optimal neural network architecture can vary depending on specific tasks and data distributions. Neural Architecture Search (NAS) is a class of techniques aimed at automatically searching for and designing neural network architectures according to the given tasks and data. Specifically, evolutionary-computation-based NAS methods are known for their strong global search capability and have aroused widespread interest in recent years. Although evolutionary-computation-based NAS has achieved success in a wide range of research and applications, it still faces bottlenecks in training and evaluating a large number of individuals during optimization. In this study, we first devise a multi-objective evolutionary NAS framework based on a weight-sharing supernet to improve the search efficiency of traditional evolutionary-computation-based NAS. This framework combines the population optimization characteristic of evolutionary algorithms with the weight-sharing ideas in one-shot models. We then design a bi-population MOEA/D algorithm based on the proposed framework to effectively solve the NAS problem. By constructing two sub-populations with different optimization objectives, the algorithm can effectively explore network architectures of various sizes in complex search spaces. An inter-population communication mechanism further enhances the algorithm’s exploratory capability, enabling it to find network architectures with uniform distribution and high diversity. Finally, we conduct performance comparison experiments on image classification datasets of different scales and complexities. Experimental results demonstrate the effectiveness of the proposed multi-objective evolutionary NAS framework and the practicality and transferability of the introduced bi-population MOEA/D-based NAS method compared to existing state-of-the-art NAS methods.

Theoretical and Legal Aspects of Regulation of Cloud Technologies

The article discusses issues of theoretical and legal aspects of the development of cloud technologies, where relations in the field of use of cloud technologies are regulated by the norms of various branches of law. The experience of implementing the concept in the field of cloud technologies has been studied, taking into account the influence of information and communication technologies on this resource. It has been determined that the development and use of online services for the distribution and processing of digital data is impossible without the use of cloud technologies, which allow companies to quickly scale their resources in accordance with business needs.

Scientific Inference with Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena

To learn about real world phenomena, scientists have traditionally used models with clearly interpretable elements. However, modern machine learning (ML) models, while powerful predictors, lack this direct elementwise interpretability (e.g. neural network weights). Interpretable machine learning (IML) offers a solution by analyzing models holistically to derive interpretations. Yet, current IML research is focused on auditing ML models rather than leveraging them for scientific inference. Our work bridges this gap, presenting a framework for designing IML methods—termed ’property descriptors’—that illuminate not just the model, but also the phenomenon it represents. We demonstrate that property descriptors, grounded in statistical learning theory, can effectively reveal relevant properties of the joint probability distribution of the observational data. We identify existing IML methods suited for scientific inference and provide a guide for developing new descriptors with quantified epistemic uncertainty. Our framework empowers scientists to harness ML models for inference, and provides directions for future IML research to support scientific understanding.

Unsupervised Imputation of Non-Ignorably Missing Data Using Importance-Weighted Autoencoders

Deep Learning (DL) methods have dramatically increased in popularity in recent years. While its initial success was demonstrated in the classification and manipulation of image data, there has been significant growth in the application of DL methods to problems in the biomedical sciences. However, the greater prevalence and complexity of missing data in biomedical datasets present significant challenges for DL methods. Here, we provide a formal treatment of missing data in the context of Variational Autoencoders (VAEs), a popular unsupervised DL architecture commonly used for dimension reduction, imputation, and learning latent representations of complex data. We propose a new VAE architecture, NIMIWAE, that is one of the first to flexibly account for both ignorable and non-ignorable patterns of missingness in input features at training time. Following training, samples can be drawn from the approximate posterior distribution of the missing data can be used for multiple imputation, facilitating downstream analyses on high dimensional incomplete datasets. We demonstrate through statistical simulation that our method outperforms existing approaches for unsupervised learning tasks and imputation accuracy. We conclude with a case study of an EHR dataset pertaining to 12,000 ICU patients containing a large number of diagnostic measurements and clinical outcomes, where many features are only partially observed.

Quantum‐Noise‐Driven Generative Diffusion Models

AbstractGenerative models realized with Machine Learning (ML) techniques are powerful tools to infer complex and unknown data distributions from a finite number of training samples in order to produce new synthetic data. Diffusion Models (DMs) are an emerging framework that have recently overcome Generative Adversarial Networks (GANs) in creating high‐quality images. Here, is proposed and discussed the quantum generalization of DMs, i.e., three Quantum‐Noise‐Driven Generative Diffusion Models (QNDGDMs) that could be experimentally tested on real quantum systems. The idea is to harness unique quantum features, in particular the non‐trivial interplay among coherence, entanglement, and noise that the currently available noisy quantum processors do unavoidably suffer from, in order to overcome the main computational burdens of classical diffusion models during inference. Hence, the suggestion is to exploit quantum noise not as an issue to be detected and solved but instead as a beneficial key ingredient to generate complex probability distributions from which a quantum processor might sample more efficiently than a classical one. Three examples of the numerical simulations are also included for the proposed approaches. The results are expected to pave the way for new quantum‐inspired or quantum‐based generative diffusion algorithms addressing tasks as data generation with widespread real‐world applications.

Bayesian Federated Learning with Hamiltonian Monte Carlo: Algorithm and Theory

This work introduces a novel and efficient Bayesian federated learning algorithm, namely, the Federated Averaging stochastic Hamiltonian Monte Carlo (FA-HMC), for parameter estimation and uncertainty quantification. We establish rigorous convergence guarantees of FA-HMC on non-iid distributed data sets, under the strong convexity and Hessian smoothness assumptions. Our analysis investigates the effects of parameter space dimension, noise on gradients and momentum, and the frequency of communication (between the central node and local nodes) on the convergence and communication costs of FA-HMC. Beyond that, we establish the tightness of our analysis by showing that the convergence rate cannot be improved even for continuous FA-HMC process. Moreover, extensive empirical studies demonstrate that FA-HMC outperforms the existing Federated Averaging-Langevin Monte Carlo (FA-LD) algorithm.

Production of activated carbons from Quercus cerris acorn shell under various experiment conditions, and their characterizations

AbstractDue to the high porosities, large surface areas, insolubilities in solutions, and unique structural and morphological structures, porous materials are utilized in various application areas such as energy conversion and storage, wastewater treatment, adsorption, catalysis and photocatalysis. In this study, activated carbons (QCACs), one type of porous materials, were synthesized from Quercus cerris acorn shells by using ZnCl2 chemical activation under various production conditions. The effects of carbonization temperature, carbonization period, and impregnation ratios on the yields, surface areas, pore developments, and N2 adsorption–desorption isotherms of activated carbons obtained were investigated in detail. The highest surface area (1751.61 m2/g) was reached when utilized at the impregnation ratio of 2.0 at 500 ℃ for 90 min. The total pore volume of QCAC increased with increasing impregnation ratio, however the micropore volume of QCAC reduced. It was found from the pore distribution data that QCACs contained mostly narrow mesopores and a little amount of micropores. Also, N2 adsorption–desorption isotherm data revealed that QCACs produced under different conditions were usually mesopore structures, and the pores were narrow slit-shaped. Moreover, the data provided from SEM, FTIR, Boehm titration, and elemental analysis gave more characterization information about QCACs synthesized.

ConvTrans-CL: Ocean time series temperature data anomaly detection based context contrast learning

Ocean temperature data anomaly detection is instrumental in monitoring environmental changes and implementing measures to alleviate adverse consequences. This holds immense importance for marine environmental observation and scientific inquiry. However, existing anomaly detection methods encounter significant challenges in extracting features from data, which severely affects the performance of anomaly detection. Existing models have limitations in capturing the local contextual distribution features and high stochastic distribution trends of ocean temperature data. Therefore, this paper introduces the ConvTrans-CL model, which integrates the Transformer encoder with causal convolution and employs a contrastive learning approach. Causal convolution extracts the distributional features of short-time subsequences within a sliding window and integrates them into the self-attention mechanism, enabling the model to focus on the localized features of the data. Contrastive learning efficiently captures the long-range dependencies of time-series data by distinguishing between pairs of subsequences with adjacent time intervals and pairs of non-adjacent subsequences. This enables the model to capture the trend of high stochasticity in the distribution of the temperature data. Finally, we select temperature data from two sea areas that are susceptible to multiple environmental factors for experiments and compare the feature extraction and anomaly detection capabilities of ConvTrans-CL with other methods, confirming the superior performance of our method.

An experimental review of the ensemble-based data stream classification algorithms in non-stationary environments

Data streams are sequences of fast-growing and high-speed data points that typically suffer from the infinite length, large volume, and specifically unstable data distribution. Ensemble learning as a prevalent classification approach is widely used in data stream mining studies. Besides the impressive performance of ensemble learning algorithms in providing a collection of diverse and accurate classifiers, they are specifically efficient in handling non-stationary data streams. Due to the component-based nature and the chance of dynamic updates for the components of the ensemble, this category is appropriate for dynamically learning the changing concepts of the data. Several review research have been conducted on the challenges of non-stationary data streams so far. None of the available surveys are dedicated to examine the effect of ensemble learning models on concept drift handling. This paper aims to provide a thorough theoretical and experimental review of the most significant ensemble-based data stream classification approaches in confronting with various types of concept drifts. In this comprehensive experimental analysis, 21 state-of-the-art ensemble algorithms are tested on 30 synthetic datasets under various types and volumes of concept drifts. This experimental analysis involves analyzing the classification accuracy, kappa score and running time of the algorithms, along with performing various statistical tests. In addition to conducting experimental analysis, a comprehensive investigation is carried out on the most significant challenges posed by non-stationary data streams, which provides the reader with valuable insights.

Uncertainty quantification for probabilistic machine learning in earth observation using conformal prediction

Machine learning is increasingly applied to Earth Observation (EO) data to obtain datasets that contribute towards international accords. However, these datasets contain inherent uncertainty that needs to be quantified reliably to avoid negative consequences. In response to the increased need to report uncertainty, we bring attention to the promise of conformal prediction within the domain of EO. Unlike previous uncertainty quantification methods, conformal prediction offers statistically valid prediction regions while concurrently supporting any machine learning model and data distribution. To support the need for conformal prediction, we reviewed EO datasets and found that only 22.5% of the datasets incorporated a degree of uncertainty information, with unreliable methods prevalent. Current open implementations require moving large amounts of EO data to the algorithms. We introduced Google Earth Engine native modules that bring conformal prediction to the data and compute, facilitating the integration of uncertainty quantification into existing traditional and deep learning modelling workflows. To demonstrate the versatility and scalability of these tools we apply them to valued EO applications spanning local to global extents, regression, and classification tasks. Subsequently, we discuss the opportunities arising from the use of conformal prediction in EO. We anticipate that accessible and easy-to-use tools, such as those provided here, will drive wider adoption of rigorous uncertainty quantification in EO, thereby enhancing the reliability of downstream uses such as operational monitoring and decision-making.

Research on Load Decomposition and Optimization of Intelligent Elderly Care Service Based on Heuristic and Event Detection Algorithm

Against the backdrop of the digital age, the openness, equality and interaction of the Internet economy have injected new vitality into China’s traditional industries. The application of big data technology, especially in information integration and analysis, has become a key force in promoting the sustainable and healthy development of the national economy. This study focuses on the “Internet +” environment, discusses the impact of the aging problem of community workers on home care services, and proposes an optimization scheme based on a heuristic algorithm. The heuristic algorithm, inspired by the foraging behavior of ants in nature, optimizes the route selection problem by simulating an ant colony to choose the path with a high concentration of pheromones and shows outstanding application potential in the field of home care. The accuracy of the event detection algorithm is directly related to the performance of the load decomposition algorithm, and the change point detection algorithm can effectively identify the change point of the probability distribution in the time series data, which provides important input data for unsupervised clustering. Advanced computer theory, including the Hidden Markov model (HMM) and swarm intelligence optimization algorithm, is used in this research. By comparing different swarm intelligence algorithms, we find that the standard Gray Wolf optimization (SGWO) model is better than the basic Gray Wolf optimization (BGWO) algorithm and the improved Gray Wolf optimization (DGWO) algorithm in terms of stability and output results. The SGWO model significantly improves the efficiency of the load decomposition algorithm, which has been verified in the application of the smart elderly care service platform. The platform not only supports the operation of related technologies and information products but also realizes the seamless integration of information among various subjects of elderly care services. In addition, the factor hidden in the Markov model that can be selectively activated effectively monitors equipment status in the Internet of Things environment, provides real-time monitoring of user consumption behavior and fault information and further enhances the quality and efficiency of smart elderly care services.

Reference Intervals of Mass Coefficients and Absolute Weights of Mini Pigs’ Organs

Introduction. Measurement of weights and calculation of mass coefficients of the laboratory animals’ internal organs are the integral stages of toxicological research. However, the absence of the correct intra-laboratory reference intervals corresponding to the normal values established for the animal population of a research laboratory could hinder the analysis of the mass coefficient changes in the experiment. For example, the data on weights and mass coefficients of mini pigs’ organs is being an issue due to its insufficient availability in the open sources. At the same time, in the literature there is a lot of such kind of data referring to the industrial pig breeds. Due to the wide use of the mini pigs in the pre-clinical research and because in the pig species taxonomy they are considered to be a weight category representing the pigs with reduced mass of the body with similar morphology of the organs and body systems, we have set an objective to determine and calculate the reference intervals of mass coefficients of internal organs relative to the mass of the body and brain, as well as the absolute weights of mini pigs’ internal organs and compare the results with the data on mini and industrial pig breeds in the literature sources.Materials and Methods. The data on 29 males and 29 females of the Vietnamese lop-bellied mini pig breed aged 6 months and older, who used to be in control groups of the various experiments held at JSC “SPA “House of Pharmacy” (Leningrad region) in the period from January 2018 to March 2024, was used to calculate the reference intervals. The statistical analysis was peRussian Federationormed using Microsoft Excel and Reference Value Advisor v2.1 software in compliance with the CLSI guidelines. Calculation of the reference intervals for all parameters was carried out by a robust method, the compliance with the normal distribution law was determined by Anderson–Darling test, in case of abnormal distribution of data, calculation of the reference interval was carried out with the Box–Cox transformation.Results. The data on the absolute weights of organs and their coefficients relative to the body and brain masses were presented in the form of the lower and upper reference limits and confidence intervals for each of the limit. The reference intervals of masses of organs across all parameters in males and females had similar limits, the most comparable values were found for the heart and brain parameters. The calculated interval of the heart absolute weight in males was 28.3–253 g, that of the brain was 46.5–93.7 g; in females these intervals were 42.9–279 g and 55.5–81.8 g respectively. Upon comparing the obtained data with the literature sources, it was found that the majority of the absolute weights of all measured mini pigs’ organs had fitted in the limits of the reference values, whereas the masses of the lungs and liver were at the lower limit or below the calculated intervals. The values of the absolute weights of most of the organs of the industrial pig breeds were higher than that of mini pigs, which was due to the difference in the body size and mass of the studied animal breeds. When comparing the mass coefficients of organs in relation to the body mass, it was found that the coefficients of most organs of the industrial breeds and mini pigs, on the contrary, were within the calculated reference intervals or slightly exceeded them.Discussion and conclusion. The calculated reference intervals of the mass coefficients of some organs in relation to the body mass, as well as the absolute weights of the spleen in mini pigs are comparable with the data available in literature for the industrial meat pig breeds. At the same time, the data available in the literature sources on the absolute weights of organs and their mass coefficients relative to the body mass of mini pigs, in most cases, have fitted into the calculated intervals. To increase the prognostic value of the obtained reference intervals, the analysis of a larger size sample is necessary.

Determining cut-off scores for simulated tasks in Brazilian Air Force military personnel

IntroductionCombat readiness assessments through simulated tasks (STs) have been developed for the Brazilian Air Force (BAF) to establish physical employment standards. Previous research has established BAF critical combat tasks with...