Continuous Domain Research Articles

Mathematical Models for the Single-Channel and Multi-Channel PMU Allocation Problem and Their Solution Algorithms

Phasor measurement units (PMUs) are deployed at power grid nodes around the transmission grid, determining precise power system monitoring conditions. In real life, it is not realistic to place a PMU at every power grid node; thus, the lowest PMU number is optimally selected for the full observation of the entire network. In this study, the PMU placement model is reconsidered, taking into account single- and multi-capacity placement models rather than the well-studied PMU placement model with an unrestricted number of channels. A restricted number of channels per monitoring device is used, instead of supposing that a PMU is able to observe all incident buses through the transmission connectivity lines. The optimization models are declared closely to the power dominating set and minimum edge cover problem in graph theory. These discrete optimization problems are directly related with the minimum set covering problem. Initially, the allocation model is declared as a constrained mixed-integer linear program implemented by mathematical and stochastic algorithms. Then, the 0/1 integer linear problem is reformulated into a non-convex constraint program to find optimality. The mathematical models are solved either in binary form or in the continuous domain using specialized optimization libraries, and are all implemented in YALMIP software in conjunction with MATLAB. Mixed-integer linear solvers, nonlinear programming solvers, and heuristic algorithms are utilized in the aforementioned software packages to locate the global solution for each instance solved in this application, which considers the transformation of the existing power grids to smart grids.

Salience Interest Option: Temporal abstraction with salience interest functions

Reinforcement Learning (RL) is a significant machine learning subfield that emphasizes learning actions based on environment to obtain optimal behavior policy. RL agents can make decisions at variable time scales in the form of temporal abstractions, also known as options. The issue of discovering options has seen a considerable research effort. Most notably, the Interest Option Critic (IOC) algorithm first extends the initial set to the interest function, providing a method for learning options specialized to certain state space regions. This approach offers a specific attention mechanism for action selection. Unfortunately, this method still suffers from the classic issues of poor data efficiency and lack of flexibility in RL when learning options end-to-end through backpropagation. This paper proposes a new approach called Salience Interest Option Critic (SIOC), which chooses subsets of existing initiation sets for RL. Specifically, these subsets are not learned by backpropagation, which is slow and tends to overfit, but through particle filters. This approach enables the rapid and flexible identification of critical subsets using only reward feedback. We conducted experiments in discrete and continuous domains, and our proposed method demonstrate higher efficiency and flexibility than other methods. The generated options are more valuable within a single task and exhibited greater interpretability and reusability in multi-task learning scenarios.

Taut cables with hanging masses: A metamaterial-like dynamic behavior

We inspect the dynamic behavior of suspended cables presenting a periodic array of scatter elements, consisting of a discrete set of masses that are hanging by means of elastic or rigid connections. By introducing some approximations, we show that the problem in the continuous domain can be brought back to an equivalent discrete problem, whose solutions depend on the variation of an effective mass. We find that the essential spectrum of the problem presents band gaps. By considering boundary conditions, eigenmodes can only be found at frequencies belonging to pass bands for the propagation problem of the unbounded system. This is no more true when a defect of periodicity is inserted, leading to the formation of localized modes. We provide a quantitative validation of these theoretical results by means of comparison with experimental tests.

Classification feature selection and dimensionality reduction based on logical binary sine-cosine function arithmetic optimization algorithm

Arithmetic optimization algorithm (AOA) is a meta-heuristic algorithm inspired by mathematical operations. AOA has been diffusely used for optimization issues on continuous domains, but few scholars have studied discrete optimization problems. In this paper, we proposed Binary AOA (BAOA) based on two strategies to handle the feature selection problem. The first strategy used S-shaped and V-shaped shift functions to map continuous variables to discrete variables. The second strategy was to combine four logical operations (AND, OR, XOR, XNOR) on the basis of the transfer function, and constructed a parameter model based on the sine and cosine function. An enhanced logic binary sine–cosine function arithmetic optimization algorithm (LBSCAOA) was proposed to realize the position update of variables. Its purpose was to improve the algorithm's global search capabilities and local exploitation capabilities. In the simulation experiments, 20 datasets were selected to testify the capability of the proposed algorithm. Since KNN had the advantages of easy understanding and low training time complexity, this classifier was selected for evaluation. The performance of the improved algorithm was comprehensively evaluated by comparing the average classification accuracy, the average number of selected features, the average fitness value and the average running time. Simulation results showed LBSCAOA with V-Shaped “V4” stood out among many improved algorithms. On the other hand, LBSCAOA with V-Shaped “V4” was used as a representative to compare with other typical feature selection algorithms to verify its competitivenes.

Hierarchical reinforcement Thompson composition

Modern real-world control problems call for continuous control domains and robust, sample efficient and explainable control frameworks. We are presenting a framework for recursively composing control skills to solve compositional and progressively complex tasks. The framework promotes reuse of skills, and as a result quick adaptability to new tasks. The decision tree can be observed, providing insight into the agents’ behavior. Furthermore, the skills can be transferred, modified or trained independently, which can simplify reward shaping and increase training speeds considerably. This paper is concerned with efficient composition of control algorithms using reinforcement learning and soft attention. Compositional and temporal abstraction is the key to improving learning and planning in reinforcement learning. Our Thompson sampling inspired soft-attention model is demonstrated to efficiently solve the composition problem.

Predictive coding with spiking neurons and feedforward gist signaling.

Predictive coding (PC) is an influential theory in neuroscience, which suggests the existence of a cortical architecture that is constantly generating and updating predictive representations of sensory inputs. Owing to its hierarchical and generative nature, PC has inspired many computational models of perception in the literature. However, the biological plausibility of existing models has not been sufficiently explored due to their use of artificial neurons that approximate neural activity with firing rates in the continuous time domain and propagate signals synchronously. Therefore, we developed a spiking neural network for predictive coding (SNN-PC), in which neurons communicate using event-driven and asynchronous spikes. Adopting the hierarchical structure and Hebbian learning algorithms from previous PC neural network models, SNN-PC introduces two novel features: (1) a fast feedforward sweep from the input to higher areas, which generates a spatially reduced and abstract representation of input (i.e., a neural code for the gist of a scene) and provides a neurobiological alternative to an arbitrary choice of priors; and (2) a separation of positive and negative error-computing neurons, which counters the biological implausibility of a bi-directional error neuron with a very high baseline firing rate. After training with the MNIST handwritten digit dataset, SNN-PC developed hierarchical internal representations and was able to reconstruct samples it had not seen during training. SNN-PC suggests biologically plausible mechanisms by which the brain may perform perceptual inference and learning in an unsupervised manner. In addition, it may be used in neuromorphic applications that can utilize its energy-efficient, event-driven, local learning, and parallel information processing nature.

Learning explanatory logical rules in non-linear domains: a neuro-symbolic approach

Deep neural networks, despite their capabilities, are constrained by the need for large-scale training data, and often fall short in generalisation and interpretability. Inductive logic programming (ILP) presents an intriguing solution with its data-efficient learning of first-order logic rules. However, ILP grapples with challenges, notably the handling of non-linearity in continuous domains. With the ascent of neuro-symbolic ILP, there’s a drive to mitigate these challenges, synergising deep learning with relational ILP models to enhance interpretability and create logical decision boundaries. In this research, we introduce a neuro-symbolic ILP framework, grounded on differentiable Neural Logic networks, tailored for non-linear rule extraction in mixed discrete-continuous spaces. Our methodology consists of a neuro-symbolic approach, emphasising the extraction of non-linear functions from mixed domain data. Our preliminary findings showcase our architecture’s capability to identify non-linear functions from continuous data, offering a new perspective in neural-symbolic research and underlining the adaptability of ILP-based frameworks for regression challenges in continuous scenarios.

An adversarial transfer learning method based on domain distribution prediction for aero-engine fault diagnosis

The utilization of transfer learning enables effective realization of the transferability of aero-engine fault diagnosis models across disparate states. However, the gradual degradation of engine performance and the complex and variable flight states lead to continuous changes in data distribution. The common transfer learning methods employ discrete divisions of data domain distributions, which are insufficient to cope with the continuous changes of the domain. To accomplish transfer learning towards a continuous and multi-dimensional target domain, we propose a continuous domain distribution adversarial network (CDDAN) based on deep domain adversarial network. The method defines the number of effective cycles directly associated with engine degradation as a continuous domain index. When the domain index is extended to multiple dimensions, a hybrid gaussian model is introduced to represent distribution prediction within multi-dimensional continuous domains. Subsequently, we validate the feasibility and superiority of our method using datasets generated based on twin-spool turbofan engines. In comparison to other transfer learning methods, the proposed method achieves superior effect in continuous domain adaptation. The experimental results demonstrate that this method exhibits adaptability to failures in the diagnosis method caused by performance degradation and changes in flight state of the aero engine, while remaining independent of target domain data annotation. This advantage becomes particularly apparent in scenarios with limited target domain data and multi-state fault diagnosis.

Cross noise level PET denoising with continuous adversarial domain generalization

Objective. Performing positron emission tomography (PET) denoising within the image space proves effective in reducing the variance in PET images. In recent years, deep learning has demonstrated superior denoising performance, but models trained on a specific noise level typically fail to generalize well on different noise levels, due to inherent distribution shifts between inputs. The distribution shift usually results in bias in the denoised images. Our goal is to tackle such a problem using a domain generalization technique. Approach. We propose to utilize the domain generalization technique with a novel feature space continuous discriminator (CD) for adversarial training, using the fraction of events as a continuous domain label. The core idea is to enforce the extraction of noise-level invariant features. Thus minimizing the distribution divergence of latent feature representation for different continuous noise levels, and making the model general for arbitrary noise levels. We created three sets of 10%, 13%–22% (uniformly randomly selected), or 25% fractions of events from 97 18F-MK6240 tau PET studies of 60 subjects. For each set, we generated 20 noise realizations. Training, validation, and testing were implemented using 1400, 120, and 420 pairs of 3D image volumes from the same or different sets. We used 3D UNet as the baseline and implemented CD to the continuous noise level training data of 13%–22% set. Main results. The proposed CD improves the denoising performance of our model trained in a 13%–22% fraction set for testing in both 10% and 25% fraction sets, measured by bias and standard deviation using full-count images as references. In addition, our CD method can improve the SSIM and PSNR consistently for Alzheimer-related regions and the whole brain. Significance. To our knowledge, this is the first attempt to alleviate the performance degradation in cross-noise level denoising from the perspective of domain generalization. Our study is also a pioneer work of continuous domain generalization to utilize continuously changing source domains.

An Improved Machine Learning Model for Pure Component Property Estimation

Information on the physicochemical properties of chemical species is an important prerequisite when performing tasks such as process design and product design. However, the lack of extensive data and high experimental costs hinder the development of prediction techniques for these properties. Moreover, accuracy and predictive capabilities still limit the scope and applicability of most property estimation methods. This paper proposes a new Gaussian process-based modeling framework that aims to manage a discrete and high-dimensional input space related to molecular structure representation with the group-contribution approach. A warping function is used to map discrete input into a continuous domain in order to adjust the correlation between different compounds. Prior selection techniques, including prior elicitation and prior predictive checking, are also applied during the building procedure to provide the model with more information from previous research findings. The framework is assessed using datasets of varying sizes for 20 pure component properties. For 18 out of the 20 pure component properties, the new models are found to give improved accuracy and predictive power in comparison with other published models, with and without machine learning.

Spatial Functional Data analysis: Irregular spacing and Bernstein polynomials

Spatial Functional Data (SFD) analysis is an emerging statistical framework that combines Functional Data Analysis (FDA) and spatial dependency modeling. Unlike traditional statistical methods, which treat data as scalar values or vectors, SFD considers data as continuous functions, allowing for a more comprehensive understanding of their behavior and variability. This approach is well-suited for analyzing data collected over time, space, or any other continuous domain. SFD has found applications in various fields, including economics, finance, medicine, environmental science, and engineering. This study proposes new functional Gaussian models incorporating spatial dependence structures, focusing on irregularly spaced data and reflecting spatially correlated curves. The model is based on Bernstein polynomial (BP) basis functions and utilizes a Bayesian approach for estimating unknown quantities and parameters. The paper explores the advantages and limitations of the BP model in capturing complex shapes and patterns while ensuring numerical stability. The main contributions of this work include the development of an innovative model designed for SFD using BP, the presence of a random effect to address associations between irregularly spaced observations, and a comprehensive simulation study to evaluate models’ performance under various scenarios. The work also presents one real application of Temperature in Mexico City, showcasing practical illustrations of the proposed model.

Large strain micromechanics of thermoplastic elastomers with random microstructures

Thermoplastic polyurethanes (TPU) are block copolymeric materials composed of plastomeric “hard” and elastomeric “soft” domains, by which they exhibit highly resilient yet dissipative large deformation features depending on volume fractions and microstructures of the two distinct domains. Here, we develop a new methodology to explore the microscopic deformation mechanisms in TPU materials with highly disordered microstructures. We propose new micromechanical models for randomly dispersed (or occluded) as well as randomly continuous hard domains (and mixtures of both dispersed and continuous hard domains), each within a continuous soft structure as widely found in representative TPU materials over a wide range of volume fractions, vhard = 26.9% to 52.2%. The micromechanical modeling results are compared to experimental data on the macroscopic large strain behaviors reported previously (Cho et al., 2017). We explore the role of the dispersed vs. continuous nature of the geometric features of the random microstructures on shape recovery and energy dissipation at the microstructural level in these phase-separated copolymeric materials.

Progressive High-Frequency Reconstruction for Pan-Sharpening with Implicit Neural Representation

Pan-sharpening aims to leverage the high-frequency signal of the panchromatic (PAN) image to enhance the resolution of its corresponding multi-spectral (MS) image. However, deep neural networks (DNNs) tend to prioritize learning the low-frequency components during the training process, which limits the restoration of high-frequency edge details in MS images. To overcome this limitation, we treat pan-sharpening as a coarse-to-fine high-frequency restoration problem and propose a novel method for achieving high-quality restoration of edge information in MS images. Specifically, to effectively obtain fine-grained multi-scale contextual features, we design a Band-limited Multi-scale High-frequency Generator (BMHG) that generates high-frequency signals from the PAN image within different bandwidths. During training, higher-frequency signals are progressively injected into the MS image, and corresponding residual blocks are introduced into the network simultaneously. This design enables gradients to flow from later to earlier blocks smoothly, encouraging intermediate blocks to concentrate on missing details. Furthermore, to address the issue of pixel position misalignment arising from multi-scale features fusion, we propose a Spatial-spectral Implicit Image Function (SIIF) that employs implicit neural representation to effectively represent and fuse spatial and spectral features in the continuous domain. Extensive experiments on different datasets demonstrate that our method outperforms existing approaches in terms of quantitative and visual measurements for high-frequency detail recovery.

Peer Learning: Learning Complex Policies in Groups from Scratch via Action Recommendations

Peer learning is a novel high-level reinforcement learning framework for agents learning in groups. While standard reinforcement learning trains an individual agent in trial-and-error fashion, all on its own, peer learning addresses a related setting in which a group of agents, i.e., peers, learns to master a task simultaneously together from scratch. Peers are allowed to communicate only about their own states and actions recommended by others: "What would you do in my situation?". Our motivation is to study the learning behavior of these agents. We formalize the teacher selection process in the action advice setting as a multi-armed bandit problem and therefore highlight the need for exploration. Eventually, we analyze the learning behavior of the peers and observe their ability to rank the agents' performance within the study group and understand which agents give reliable advice. Further, we compare peer learning with single agent learning and a state-of-the-art action advice baseline. We show that peer learning is able to outperform single-agent learning and the baseline in several challenging discrete and continuous OpenAI Gym domains. Doing so, we also show that within such a framework complex policies from action recommendations beyond discrete action spaces can evolve.

Bayesian spatio-temporal statistical modeling of violent-related fatality in western and central Africa

Fatality arising from violent events is a critical public health problem in Africa. Although numerous studies on crime and violent events have been conducted, adequate attention has not been given to the distribution of fatalities arising from these events. This study unraveled the spatio-temporal pattern of fatality from violent events in Western and Central Africa. A two-component spatio-temporal zero-inflated model on a continuous spatial domain within a Bayesian framework was adopted. The stochastic partial differential equation was used to quantify the continuous pattern and make projections in unsampled regions. Fatality data from 1997 to 2021 was obtained from the Armed Conflict Location and Event Data Project (ACLED). Findings from the result revealed a spatial and temporal divide in the prevalence of fatality in the study region. Between the years 1997 and 2010, fatality from violence was most prevalent in Central Africa, whereas in more recent years, it was most prevalent in Western Africa. The posterior predictive probabilities of fatality occurrence due to violent events in Nigeria and Cameroon were highest and above 0.6, and the probability of more than one death per violent event is highest in Angola and Chad with probability 0.2. On violent event type, findings showed that suicide bombs had the highest likelihood of fatality occurrence whereas the event of violent non-state actors overtaking territory had the highest impact on the likelihood of multiple fatality counts. Among the armed actors who participated in violent events, armed religious groups were linked to the highest likelihood of fatality occurrence whereas Military forces were linked to the highest likelihood of multiple fatality counts per event. The finding also revealed that there is a higher likelihood of multiple fatalities in the Winter temperate season. These findings could be used for planning and policy design geared towards mitigating fatality and providing a guide towards resource distribution to support the affected communities.

Approximate and Memorize (A&M) : Settling opposing views in replay-based continuous unsupervised domain adaptation

Continuous Unsupervised Domain Adaptation (CUDA) can alleviate deep learning models’ performance degradation on out-of-distribution data. However, low stability, the erosion of past knowledge while adapting to new domains, remains a major challenge. Traditional proposed solutions such as (1) Approximation, or (2) Memorization fail to harmonize system’s computational load, memory resources and stability. In response, we introduce Approximate and Memorize (A&M). Compared to traditional approximation methods, A&M utilizes factorized generative models to mitigate modal collapse, offering improved computational efficiency and training stability. Moreover, compared to traditional memorization approaches, A&M boosts memory efficiency by learning compressed representations of past information rather than storing it in the raw format. A&M shows 50% less forgetting for problems with constrained memory and an extended number of domains, making A&M ideal for real-world systems due to its scalable and parallel design.

A LOPEZ-ESCOBAR THEOREM FOR CONTINUOUS DOMAINS

Abstract We prove an effective version of the Lopez-Escobar theorem for continuous domains. Let $Mod(\tau )$ be the set of countable structures with universe $\omega $ in vocabulary $\tau $ topologized by the Scott topology. We show that an invariant set $X\subseteq Mod(\tau )$ is $\Pi ^0_\alpha $ in the Borel hierarchy of this topology if and only if it is definable by a $\Pi ^p_\alpha $ -formula, a positive $\Pi ^0_\alpha $ formula in the infinitary logic $L_{\omega _1\omega }$ . As a corollary of this result we obtain a new pullback theorem for positive computable embeddings: Let $\mathcal {K}$ be positively computably embeddable in $\mathcal {K}'$ by $\Phi $ , then for every $\Pi ^p_\alpha $ formula $\xi $ in the vocabulary of $\mathcal {K}'$ there is a $\Pi ^p_\alpha $ formula $\xi ^{*}$ in the vocabulary of $\mathcal {K}$ such that for all $\mathcal {A}\in \mathcal {K}$ , $\mathcal {A}\models \xi ^{*}$ if and only if $\Phi (\mathcal {A})\models \xi $ . We use this to obtain new results on the possibility of positive computable embeddings into the class of linear orderings.

Consecutive knowledge meta-adaptation learning for unsupervised medical diagnosis

Deep learning-based computer-aided diagnosis has garnered significant attention in both academic research and clinical applications. Due to the challenges in collecting well-labeled clinical data, unsupervised domain adaptation methods have become widely used in medical image analysis. However, existing unsupervised domain adaptation methods in the clinic field fail to adapt to scenarios where new, unlabeled target data continually arrive in an online manner. To tackle this real-world clinical challenge, a meta-adaptation framework is innovatively developed, termed consecutive lesion knowledge meta-adaptation, primarily comprising semantic adaptation and representation adaptation phases. Specifically, the semantic knowledge learned from the source lesion domain will be transferred to the consecutive target lesion in the semantic adaptation phase. To further align the transferable representation knowledge across the source and multiple target lesion domains, the feature-extractor is optimized in the representation adaptation phase. Furthermore, to mitigate catastrophic forgetting during continuous unsupervised domain adaptation, a domain-quantizer is designed to preserve utility knowledge from previously learned target lesion domains. Moreover, as semantic knowledge is continuously evolving and artificially designed kernels struggle to model such complex distributions, a self-adaptive kernel is designed to flexibly measure the shift between the source and consecutive target lesion domains during the semantic adaptation phase. Experimental results on medical image classification tasks demonstrate the effectiveness of the proposed method.

Adaptive fuzzy-evidential classification based on association rule mining

As one of the most promising classification approaches, association classification (AC) integrates data classification and association discovery techniques for generating a compact set of classification association rules. Recently, the fuzzy set and evidence theories are successively applied into AC in order to improve the classification performance in terms of accuracy and interpretability. However, from the perspectives of applicability and universality, there still exists two important issues in the current AC framework. On one hand, several key parameters, such as the number of fused rules in classification and the minimum support threshold in association discovery, are difficult to be accurately predefined in practice. On the other hand, the fixed grid-based fuzzy partition is not benefit to adapt for those datasets with large number of features. In this paper, an association rule-based adaptive fuzzy-evidential classification framework (AR-AFEC) is developed for overcoming the above limitations. To do so, an optimal rule fusion strategy and a dynamic minimum support threshold setting scheme are proposed for adaptively learning the parameters during classification and association mining respectively. In addition, an entropy-based trapezoidal fuzzy partition technique is proposed to adaptively obtain the fuzzy sets defined on each continuous feature domain. Experiments on 26 benchmark datasets and a human activity recognition application demonstrate that the proposal can achieve better accuracy than some state-of-the-art rule-based classification approaches, using less rules with more general structure.

SSER: Semi-Supervised Emotion Recognition based on Triplet Loss and pseudo label

Recently, emotion recognition from facial expressions has achieved unprecedented accuracy with the development of deep learning. Despite this progress, most existing emotion recognition methods are supervised and thus require extensive annotation. This issue is particularly pronounced in continuous domain datasets where annotation costs are very high. Furthermore, discrete domain datasets containing specific poses are too uniform to reflect complex and actual emotions. Existing methods that employ classification loss pay little attention to image similarity, making it difficult to distinguish similar emotions. To improve the learning ability for image similarity and reduce the annotation cost of continuous domain datasets, this research proposes a Semi-Supervised Emotion Recognition (SSER) method, which incorporates Activation-matrix Triplet loss (AMT loss) and pseudo label with Complementary Information (CI label). Specifically, the AMT loss is constructed by encoding multiple activation channels of an image as a matrix, which are utilized to capture the image similarity. The CI label firstly adopts the coupling effect of the complementary information from images and the multi-stage model for SSL to obtain high-confidence pseudo-labels. Then, entropy minimization and consistency regularization are used to improve the accuracy of pseudo labels. The SSER is evaluated on continuous domain datasets (AFEW-VA and AFF-Wild) and discrete domain datasets (FER2013 and CK+). The experimental results demonstrate that the SSER combined with AMT loss and CI label makes improvement for emotion recognition on continuous domain datasets, meanwhile the SSER is also desirable and effective for emotion recognition on discrete domain datasets.