General Problem Research Articles

Dissipative structures for the system of Moore–Gibson–Thompson thermoelasticity in the whole space

We investigate the dissipative structure for the system of Moore–Gibson–Thompson (MGT) thermoelasticity in the whole space. To analyze the dissipative structure, it is very useful to rewrite the equations into a symmetric hyperbolic system and apply the so‐called stability condition. When we rewrite our system into the symmetric hyperbolic form in the multidimensional case, it is important to take the constraint conditions into account. Indeed, the stability condition with the constraint conditions guarantees the dissipative property for our system in some cases. In this paper, we introduce the stability condition with constraints for the general problem and apply this argument to the system of MGT thermoelasticity. Furthermore, we discuss the optimality of the decay estimates we obtain, together with the no regularity‐loss phenomenon.

Out-of-Domain Generalization From a Single Source: An Uncertainty Quantification Approach.

We are concerned with a worst-case scenario in model generalization, in the sense that a model aims to perform well on many unseen domains while there is only one single domain available for training. We propose Meta-Learning based Adversarial Domain Augmentation to solve this Out-of-Domain generalization problem. The key idea is to leverage adversarial training to create "fictitious" yet "challenging" populations, from which a model can learn to generalize with theoretical guarantees. To facilitate fast and desirable domain augmentation, we cast the model training in a meta-learning scheme and use a Wasserstein Auto-Encoder to relax the widely used worst-case constraint. We further improve our method by integrating uncertainty quantification for efficient domain generalization. Extensive experiments on multiple benchmark datasets indicate its superior performance in tackling single domain generalization.

Perturbing masses: A study of centered co-circular central configurations in power-law n-body problems

This research investigates centered co-circular central configurations in the general power-law potential n-body problem. Firstly, there are no centered co-circular central configurations exist when precisely n−2 of the masses are equal; secondly, unless all masses are equal, no such configurations exist when masses can be divided into two sets of equal masses; thirdly, no such configurations exist when masses is a mix of equal masses and some heavier (lighter) ones, arranged such that the heavier (lighter) ones are non-adjacent.

Learning physics-based reduced-order models from data using nonlinear manifolds.

We present a novel method for learning reduced-order models of dynamical systems using nonlinear manifolds. First, we learn the manifold by identifying nonlinear structure in the data through a general representation learning problem. The proposed approach is driven by embeddings of low-order polynomial form. A projection onto the nonlinear manifold reveals the algebraic structure of the reduced-space system that governs the problem of interest. The matrix operators of the reduced-order model are then inferred from the data using operator inference. Numerical experiments on a number of nonlinear problems demonstrate the generalizability of the methodology and the increase in accuracy that can be obtained over reduced-order modeling methods that employ a linear subspace approximation.

Sparsity-Constrained Invariant Risk Minimization for Domain Generalization With Application to Machinery Fault Diagnosis Modeling.

Machine learning has been widely applied to study AI-informed machinery fault diagnosis. This work proposes a sparsity-constrained invariant risk minimization (SCIRM) framework, which develops machine-learning models with better generalization capacities for environmental disturbances in machinery fault diagnosis. The SCIRM is built by innovating the optimization formulation of the recently proposed invariant risk minimization (IRM) and its variants through the integration of sparsity constraints. We prove that if a sparsity measure is differentiable, scale invariant, and semistrictly quasi-convex, the SCIRM can be guaranteed to solve the domain generalization problem based on a few predefined problem settings. We mathematically derive a family of such sparsity measures. A practical process of implementing the SCIRM for machinery fault diagnosis tasks is offered. We first verify our theoretical exploration of the SCIRM by using simulation data. We further compare SCIRM with a set of state-of-the-art methods by using real machinery fault data collected under a variety of working conditions. The computational results confirm that the machinery fault diagnosis model developed by the SCIRM offers a higher generalization capacity and performs better than the other benchmarks across the different testing datasets.

Pseudo-Data Based Self-Supervised Federated Learning for Classification of Histopathological Images.

Computer-aided diagnosis (CAD) can help pathologists improve diagnostic accuracy together with consistency and repeatability for cancers. However, the CAD models trained with the histopathological images only from a single center (hospital) generally suffer from the generalization problem due to the straining inconsistencies among different centers. In this work, we propose a pseudo-data based self-supervised federated learning (FL) framework, named SSL-FT-BT, to improve both the diagnostic accuracy and generalization of CAD models. Specifically, the pseudo histopathological images are generated from each center, which contain both inherent and specific properties corresponding to the real images in this center, but do not include the privacy information. These pseudo images are then shared in the central server for self-supervised learning (SSL) to pre-train the backbone of global mode. A multi-task SSL is then designed to effectively learn both the center-specific information and common inherent representation according to the data characteristics. Moreover, a novel Barlow Twins based FL (FL-BT) algorithm is proposed to improve the local training for the CAD models in each center by conducting model contrastive learning, which benefits the optimization of the global model in the FL procedure. The experimental results on four public histopathological image datasets indicate the effectiveness of the proposed SSL-FL-BT on both diagnostic accuracy and generalization.

Social Network Community Detection to Deal with Gray-Sheep and Cold-Start Problems in Music Recommender Systems

Information from social networks is currently being widely used in many application domains, although in the music recommendation area, its use is less common because of the limited availability of social data. However, most streaming platforms allow for establishing relationships between users that can be leveraged to address some drawbacks of recommender systems. In this work, we take advantage of the social network structure to improve recommendations for users with unusual preferences and new users, thus dealing with the gray-sheep and cold-start problems, respectively. Since collaborative filtering methods base the recommendations for a given user on the preferences of his/her most similar users, the scarcity of users with similar tastes to the gray-sheep users and the unawareness of the preferences of the new users usually lead to bad recommendations. These general problems of recommender systems are worsened in the music domain, where the popularity bias drawback is also present. In order to address these problems, we propose a user similarity metric based on the network structure as well as on user ratings. This metric significantly improves the recommendation reliability in those scenarios by capturing both homophily effects in implicit communities of users in the network and user similarity in terms of preferences.

Invariant Feature Learning Based on Causal Inference from Heterogeneous Environments

Causality has become a powerful tool for addressing the out-of-distribution (OOD) generalization problem, with the idea of invariant causal features across domains of interest. Most existing methods for learning invariant features are based on optimization, which typically fails to converge to the optimal solution. Therefore, obtaining the variables that cause the target outcome through a causal inference method is a more direct and effective method. This paper presents a new approach for invariant feature learning based on causal inference (IFCI). IFCI detects causal variables unaffected by the environment through the causal inference method. IFCI focuses on partial causal relationships to work efficiently even in the face of high-dimensional data. Our proposed causal inference method can accurately infer causal effects even when the treatment variable has more complex values. Our method can be viewed as a pretreatment of data to filter out variables whose distributions change between different environments, and it can then be combined with any learning method for classification and regression. The result of empirical studies shows that IFCI can detect and filter out environmental variables affected by the environment. After filtering out environmental variables, even a model with a simple structure and common loss function can have strong OOD generalization capability. Furthermore, we provide evidence to show that classifiers utilizing IFCI achieve higher accuracy in classification compared to existing OOD generalization algorithms.

Domain Generalization in Time Series Forecasting

Domain generalization aims to design models that can effectively generalize to unseen target domains by learning from observed source domains. Domain generalization poses a significant challenge for time series data, due to varying data distributions and temporal dependencies. Existing approaches to domain generalization are not designed for time series data, which often results in suboptimal or unstable performance when confronted with diverse temporal patterns and complex data characteristics. We propose a novel approach to tackle the problem of domain generalization in time series forecasting. We focus on a scenario where time series domains share certain common attributes and exhibit no abrupt distribution shifts. Our method revolves around the incorporation of a key regularization term into an existing time series forecasting model: domain discrepancy regularization . In this way, we aim to enforce consistent performance across different domains that exhibit distinct patterns. We calibrate the regularization term by investigating the performance within individual domains and propose the domain discrepancy regularization with domain difficulty awareness . We demonstrate the effectiveness of our method on multiple datasets, including synthetic and real-world time series datasets from diverse domains such as retail, transportation, and finance. Our method is compared against traditional methods, deep learning models, and domain generalization approaches to provide comprehensive insights into its performance. In these experiments, our method showcases superior performance, surpassing both the base model and competing domain generalization models across all datasets. Furthermore, our method is highly general and can be applied to various time series models.

A Deep Learning Based Platform for Remote Sensing Images Change Detection Integrating Crowdsourcing and Active Learning.

Remote sensing images change detection technology has become a popular tool for monitoring the change type, area, and distribution of land cover, including cultivated land, forest land, photovoltaic, roads, and buildings. However, traditional methods which rely on pre-annotation and on-site verification are time-consuming and challenging to meet timeliness requirements. With the emergence of artificial intelligence, this paper proposes an automatic change detection model and a crowdsourcing collaborative framework. The framework uses human-in-the-loop technology and an active learning approach to transform the manual interpretation method into a human-machine collaborative intelligent interpretation method. This low-cost and high-efficiency framework aims to solve the problem of weak model generalization caused by the lack of annotated data in change detection. The proposed framework can effectively incorporate expert domain knowledge and reduce the cost of data annotation while improving model performance. To ensure data quality, a crowdsourcing quality control model is constructed to evaluate the annotation qualification of the annotators and check their annotation results. Furthermore, a prototype of automatic detection and crowdsourcing collaborative annotation management platform is developed, which integrates annotation, crowdsourcing quality control, and change detection applications. The proposed framework and platform can help natural resource departments monitor land cover changes efficiently and effectively.

The Effect Of Optimizers On The Generalizability Additive Neural Attention For Customer Support Twitter Dataset In Chatbot Application

When optimizing the performance of neural network-based chatbots, determining the optimizer is one of the most important aspects. Optimizers primarily control the adjustment of model parameters such as weight and bias to minimize a loss function during training. Adaptive optimizers such as ADAM have become a standard choice and are widely used for their invariant parameter updates' magnitudes concerning gradient scale variations, but often pose generalization problems. Alternatively, Stochastic Gradient Descent (SGD) with Momentum and the extension of ADAM, the ADAMW, offers several advantages. This study aims to compare and examine the effects of these optimizers on the chatbot CST dataset. The effectiveness of each optimizer is evaluated based on its sparse-categorical loss during training and BLEU in the inference phase, utilizing a neural generative attention-based additive scoring function. Despite memory constraints that limited ADAMW to ten epochs, this optimizer showed promising results compared to configurations using early stopping techniques. SGD provided higher BLEU scores for generalization but was very time-consuming. The results highlight the importance of finding a balance between optimization performance and computational efficiency, positioning ADAMW as a promising alternative when training efficiency and generalization are primary concerns.

Reentrant open shop scheduling problem with time lags and no-wait constraints

In this work, we consider the two-machine reentrant open shop scheduling problem with no-wait and exact time lags constraints. The objective is to minimize the makespan. We first prove that this problem is NP-hard in the strong sense, then we provide some polynomial sub-problems. To solve the general problem, a heuristic and metaheuristics with hybridizations are proposed with numerical experiments.

A One-Class Classifier for the Detection of GAN Manipulated Multi-Spectral Satellite Images

The current image generative models have achieved a remarkably realistic image quality, offering numerous academic and industrial applications. However, to ensure these models are used for benign purposes, it is essential to develop tools that definitively detect whether an image has been synthetically generated. Consequently, several detectors with excellent performance in computer vision applications have been developed. However, these detectors cannot be directly applied as they areto multi-spectral satellite images, necessitating the training of new models. While two-class classifiers generally achieve high detection accuracies, they struggle to generalize to image domains and generative architectures different from those encountered during training. In this paper, we propose a one-class classifier based on Vector Quantized Variational Autoencoder 2 (VQ-VAE 2) features to overcome the limitations of two-class classifiers. We start by highlighting the generalization problem faced by binary classifiers. This was demonstrated by training and testing an EfficientNet-B4 architecture on multiple multi-spectral datasets. We then illustrate that the VQ-VAE 2-based classifier, which was trained exclusively on pristine images, could detect images from different domains and generated by architectures not encountered during training. Finally, we conducted a head-to-head comparison between the two classifiers on the same generated datasets, emphasizing the superior generalization capabilities of the VQ-VAE 2-based detector, wherewe obtained a probability of detection at a 0.05 false alarm rate of 1 for the blue and red channels when using the VQ-VAE 2-based detector, and 0.72 when we used the EfficientNet-B4 classifier.

A comparison of alternative models for solving a non-linear single plant Hydro Unit Commitment problem

A wide range of real world optimization problems involves continuous decisions and non-linearities. Each non-linear component of such problems can be modeled either linearly or non-linearly, considering or not additional integer variables. This results into different modeling choices that can drastically impact the solution time and quality. In this paper, we evaluate representative modeling alternatives, including common models from the literature as well as new models featuring less common functions. The single plant Hydro Unit Commitment problem (1-HUC) is the considered non-linear use case. Among the non-linearities of the 1-HUC, we focus on those involved in the power production, more precisely the head effect and the turbines efficiency. The power is defined as a two-dimensional non-convex and non-concave function of the water flow and head decision variables, the latter being itself a one-dimensional concave function of the turbined volume. We consider both the general problem and a common special case, assuming that the water head is fixed. Several available solvers are used for each non-linear model and the best virtual solver is retained to focus on the model capabilities rather than on the solver performance. Based on the numerical experiments, three models stand out as the most efficient in terms of computational time, solution quality and feasibility, sometimes in a counter-intuitive manner. For each of these models, a solver is highlighted as the most adequate.

Reconstruction techniques for quantum trees

The inverse problem of recovery of a potential on a quantum tree graph from the Weyl matrix given at a number of points is considered. A method for its numerical solution is proposed. The overall approach is based on the leaf peeling method combined with Neumann series of Bessel functions (NSBF) representations for solutions of Sturm–Liouville equations. In each step, the solution of the arising inverse problems reduces to dealing with the NSBF coefficients. The leaf peeling method allows one to localize the general inverse problem to local problems on sheaves, while the approach based on the NSBF representations leads to splitting the local problems into two‐spectrum inverse problems on separate edges and reduces them to systems of linear algebraic equations for the NSBF coefficients. Moreover, the potential on each edge is recovered from the very first NSBF coefficient. The proposed method leads to an efficient numerical algorithm that is illustrated by numerical tests.

Generalisation challenges in deep learning models for medical imagery: insights from external validation of COVID-19 classifiers

The generalisability of deep neural network classifiers is emerging as one of the most important challenges of our time. The recent COVID-19 pandemic led to a surge of deep learning publications that proposed novel models for the detection of COVID-19 from chest x-rays (CXRs). However, despite the many outstanding metrics reported, such models have failed to achieve widespread adoption into clinical settings. The significant risk of real-world generalisation failure has repeatedly been cited as one of the most critical concerns, and is a concern that extends into general medical image modelling. In this study, we propose a new dataset protocol and, using this, perform a thorough cross-dataset evaluation of deep neural networks when trained on a small COVID-19 dataset, comparable to those used extensively in recent literature. This allows us to quantify the degree to which these models can generalise when trained on challenging, limited medical datasets. We also introduce a novel occlusion evaluation to quantify model reliance on shortcut features. Our results indicate that models initialised with ImageNet weights then fine-tuned on small COVID-19 datasets, a standard approach in the literature, facilitate the learning of shortcut features, resulting in unreliable, poorly generalising models. In contrast, pre-training on related CXR imagery can stabilise cross-dataset performance. The CXR pre-trained models demonstrated a significantly smaller generalisation drop and reduced feature dependence outwith the lung region, as indicated by our occlusion test. This paper demonstrates the challenging problem of model generalisation, and the need for further research on developing techniques that will produce reliable, generalisable models when learning with limited datasets.

Separators in Continuous Petri Nets

Leroux has proved that unreachability in Petri nets can be witnessed by a Presburger separator, i.e. if a marking $\vec{m}_\text{src}$ cannot reach a marking $\vec{m}_\text{tgt}$, then there is a formula $\varphi$ of Presburger arithmetic such that: $\varphi(\vec{m}_\text{src})$ holds; $\varphi$ is forward invariant, i.e., $\varphi(\vec{m})$ and $\vec{m} \rightarrow \vec{m}'$ imply $\varphi(\vec{m}'$); and $\neg \varphi(\vec{m}_\text{tgt})$ holds. While these separators could be used as explanations and as formal certificates of unreachability, this has not yet been the case due to their worst-case size, which is at least Ackermannian, and the complexity of checking that a formula is a separator, which is at least exponential (in the formula size). We show that, in continuous Petri nets, these two problems can be overcome. We introduce locally closed separators, and prove that: (a) unreachability can be witnessed by a locally closed separator computable in polynomial time; (b) checking whether a formula is a locally closed separator is in NC (so, simpler than unreachability, which is P-complete). We further consider the more general problem of (existential) set-to-set reachability, where two sets of markings are given as convex polytopes. We show that, while our approach does not extend directly, we can efficiently certify unreachability via an altered Petri net.

Confidence partitioning sampling filtering

AbstractThe confidence partitioning sampling filter (CPSF) method proposed in this paper is a novel approach for solving the generic nonlinear filtering problem. First, the confidence probability space (CPS) is defined, which restricts the state transition in a bounded and closed state space in the recursive Bayesian filtering. In the posterior CPS, the weighted grid samples, represented the posterior PDF, are obtained by using the partitioning sampling technique (PST). Each weighted grid sample is treated as an impulse function. The approximate expression of the posterior PDF, as key for the PST implementation, is obtained by using the properties of the impulse function in the integral operation. By executing the selection of the CPS and the PST step repeatedly, the CPSF framework is formed to achieve the approximation of the recursive Bayesian filtering. Second, the difficulty of the CPSF framework implementation lies in obtaining the real posterior CPS. Therefore, the space intersection (SI) method is suggested to obtain the approximate posterior CPS. On this basis, the SI_CPSF algorithm, as an executable algorithm, is formed to solve the generic nonlinear filtering problem. Third, the approximate error between the CPSF framework and the recursive Bayesian filter is analyzed theoretically. The consistency of the CPSF framework to the recursive Bayesian filter is proved. Finally, the performances of the SI_CPSF algorithm, including robustness, accuracy and efficiency, are evaluated using four representative simulation experiments. The simulation results showed that SI_CSPF requires far less samples than particle filter (PF) under the same accuracy. Its computation is on average one order of magnitude less than that of the PF. The robustness of the proposed algorithm is also evaluated in the simulations.

Domain Generalization by Functional Regression

The problem of domain generalization is to learn, given data from different source distributions, a model that can be expected to generalize well on new target distributions which are only seen through unlabeled samples. In this paper, we study domain generalization as a problem of functional regression. Our concept leads to a new algorithm for learning a linear operator from marginal distributions of inputs to the corresponding conditional distributions of outputs given inputs. Our algorithm allows a source distribution-dependent construction of reproducing kernel Hilbert spaces for prediction, and, satisfies finite sample error bounds for the idealized risk. Numerical implementations and source code are available 1 .

The normality/emergency imaginary, contingency and political possibility: Analysing the UK pandemic response

AbstractThe outside of normality and normal politics is commonly referred to as emergencies, crises and disasters. The paper describes and analyses this normality/emergency imaginary by relating it to questions about order, necessity and contingency. The paper draws upon Sergei Prozorov's work on order and its excess to examine the shift in the United Kingdom pandemic response from recommendations to mandates and regulations in late March 2020. It is argued that the normality/emergency imaginary transposes the more general problem of necessity and contingency into a less complex one, thus providing a solution to questions of order, but that this displaces and eludes the important questions of contingency as a precondition for politics. Specifically, it reduces questions of order and contingency to a choice between normality or emergency where normality is rendered just and emergency measures come to be seen as necessary and un‐political. Indeed, the normality/emergency imaginary, and in particular the assumption that it is analogous to the order/contingency problem, makes it difficult to mount a political critique of emergency measures that does not reproduce and reaffirm the problem that motivates it.