Source Instances Research Articles

A two-layer regression network for robust and accurate domain adaptation

Naturally, classification models suffer from severe performance degradation when they are tested on datasets different from the ones used in training. Unsupervised domain adaptation helps to improve the generalizability of a pre-trained model by transferring knowledge from the labeled source domain (i.e., training dataset) to the unlabeled target domain (i.e., test dataset). The typical way of domain adaptation is to align the data distributions in embedding spaces between source and target domains. However, most existing works only enhance cross-domain consistency in the latent space disregarding the impact of source samples on the target classification task. Besides, discovering a subspace shared by two domains first and then training a transfer classifier separately hardly ensures that the obtained target features are suited for the classification model. To address these issues, in this work, we propose a two-layer regression network for domain adaptation (TRN-DA). TRN-DA learns class-wise domain invariant feature representations by jointly optimizing three tasks: supervised classification of source domain, unsupervised reconstruction of target domain, and alignment of cross-domain distributions. Furthermore, we introduce the concept of weight (importance) for source instances so that the resulting classifier can adapt well to the target domain. We formulate the domain adaptation problem as a unified optimization problem and solve it in an iterative way. Experimental results on different cross-domain image classification tasks demonstrate the encouraging performance of our TRN-DA compared to several recent state-of-the-art methods.

Kernel Extreme Learning Machine with Discriminative Transfer Feature and Instance Selection for Unsupervised Domain Adaptation

The goal of domain adaptation (DA) is to develop a robust decision model on the source domain effectively generalize to the target domain data. State-of-the-art domain adaptation methods typically focus on finding an optimal inter-domain invariant feature representation or helpful instances from the source domain. In this paper, we propose a kernel extreme learning machine with discriminative transfer features and instance selection (KELM-DTF-IS) for unsupervised domain adaptation tasks, which consists of two steps: discriminative transfer feature extraction and classification with instance selection. At the feature extraction stage, we extend cross domain mean approximation(CDMA) by incorporating a penalty term and develop discriminative cross domain mean approximation (d-CDMA) to optimize the category separability between instances. Subsequently, d-CDMA is integrated into kernel ELM-AutoEncoder(KELM-AE) for extracting inter-domain invariant features. During the classification process, our approach uses CDMA metrics to compute a weights to each source instances based on their impact in reducing distribution differences between domains. Instances with a greater effect receive higher weights and vice versa. These weights are then used to distinguish and select source domain instances before incorporating them into weight KELM for proposing an adaptive classifier. Finally, we apply our approach to conduct classification experiments on publicly available domain adaptation datasets, and the results demonstrate its superiority over KELM and numerous other domain adaptation approaches.

Weighted unsupervised domain adaptation considering geometry features and engineering performance of 3D design data

The product design process in manufacturing involves iterative design modeling and analysis to achieve the target engineering performance, but such an iterative process is time consuming and computationally expensive. Recently, deep learning-based engineering performance prediction models have been proposed to accelerate design optimization. However, they only guarantee predictions on training data and may be inaccurate when applied to new domain data. In particular, 3D design data have complex features, which means domains with various distributions exist. Thus, the utilization of deep learning has limitations due to the heavy data collection and training burdens. We propose a bi-weighted unsupervised domain adaptation approach that considers the geometry features and engineering performance of 3D design data. The proposed method is the first to apply domain adaptation to deep learning-based engineering performance prediction to overcome the problem of insufficient industrial data. In addition, we present a source instance weighting method called the bi-weighting strategy to consider the geometry features and engineering performance of 3D design data and avoid negative transfer for accurate prediction. Domain-invariant features can be extracted through an adversarial training strategy by using hypothesis discrepancy, and a multi-output regression task can be performed with the extracted features to predict the engineering performance. The proposed model is tested on a wheel impact analysis problem to predict the magnitude of the maximum von Mises stress and the corresponding location of 3D road wheels. This mechanism can reduce the target risk for unlabeled target domains on the basis of weighted multi-source domain knowledge and can efficiently replace conventional finite element analysis.

Attention guided partial domain adaptation for interpretable transfer diagnosis of rotating machinery

Domain adaptation (DA) is a well-established method to tackle the transfer diagnosis of rotating machinery. However, current DA methods encounter challenges in partial-set transfer diagnosis scenarios, where the target domain only possesses a subset of the source fault classes, due to several limitations: (1) assuming identical fault classes between the target and source domains, (2) neglecting the fine-grained differences among individual instances, and (3) lacking interpretability for transfer logic. To address these problems, this paper proposes an attention-guided partial domain adaptation (AGPDA) method for interpretable transfer diagnosis of rotating machinery. Firstly, to exclude outlier classes in the source domain, a transferability evaluator is presented to evaluate the potential benefits of source instances for domain alignment. Secondly, a novel cross-domain attention (CDA) mechanism is further developed, which leverages instance-wise attention to highlight relevant samples, enabling differentiated and effective knowledge transfer to various target samples. Finally, the CDA mechanism is implemented via a new interpretable linear cosine attention, which is specially designed to facilitate the exploration of the transfer logic. Comparison experiments demonstrate the superiority of AGPDA in both closed-set and partial-set transfer diagnosis. Furthermore, dual-perspective visualization of CDA reveals that target fault recognition primarily relies on source samples in the same class.

Meta-Learning With Distributional Similarity Preference for Few-Shot Fault Diagnosis Under Varying Working Conditions.

Few-shot fault diagnosis is a challenging problem for complex engineering systems due to the shortage of enough annotated failure samples. This problem is increased by varying working conditions that are commonly encountered in real-world systems. Meta-learning is a promising strategy to solve this point, open issues remain unresolved in practical applications, such as domain adaptation, domain generalization, etc. This article attempts to improve domain adaptation and generalization by focusing on the distribution-shift robustness of meta-learning from the task generation perspective. In fact, few-shot fault diagnosis under varying working conditions allows to address the distribution shift problem in a natural way. An unsupervised across-tasks meta-learning strategy with distributional similarity preference is proposed, where the core is the distribution-distance-weighting mechanism. Differently from the naive random meta-train task generation strategy used in existing meta-learning methods, the source instances that present a more similar distribution with respect to the target instances gain larger weightings in the task generation. This strategy leads to a meta-task training set that is enough diverse, and at the same time can be easily learned due to the distribution similarity features of the source tasks. The proposed method introduces the concept of maximum mean discrepancy that is applied to derive the distribution distance of the measurements. Moreover, a model-agnostic meta-learning is applied to realize few-shot fault diagnosis under varying working conditions. The proposed solutions are verified and compared by considering two public datasets used for bearing fault diagnosis. The results show that the proposed strategy outperforms different related few-shot fault diagnosis methods under varying working conditions. Moreover, it is thus proved that, meta-learning with distribution similarity feature represents an effective approach for domain adaptation and generalization.

Tighter bounds on the minimum broadcast time

Given a connected graph and a subset of its vertices referred to as the sources, the minimum broadcast time problem asks for the shortest time necessary for communicating a message from the sources to all other vertices in the graph. Information exchange is possible only between neighbors, and each vertex can transmit the message to at most one neighbor at a time. Since early works on complexity theory, the problem has been known to be NP-hard. Contributions from the current text to the understanding of the minimum broadcast time problem are threefold. Through considerations of the shortest distances between sources and other vertices, a new lower bound on the broadcast time is derived. Analytical expressions of this bound are given in the single source instances of several graph classes. Fast procedures for computing upper bounds are studied next, including both the construction of feasible solutions, and the improvement of existing ones. Finally, with a focus on a new stable-set interpretation of the problem, integer programming formulations are studied, and for their theoretical interest, associated facet-defining valid inequalities are given. The computational performance of the novel methodology is evaluated in numerical experiments applied to standard benchmark instances and to instances larger than those studied in preceding recent works.

Contrastive Learning Assisted-Alignment for Partial Domain Adaptation.

This work addresses unsupervised partial domain adaptation (PDA), in which classes in the target domain are a subset of the source domain. The key challenges of PDA are how to leverage source samples in the shared classes to promote positive transfer and filter out the irrelevant source samples to mitigate negative transfer. Existing PDA methods based on adversarial DA do not consider the loss of class discriminative representation. To this end, this article proposes a contrastive learning-assisted alignment (CLA) approach for PDA to jointly align distributions across domains for better adaptation and to reweight source instances to reduce the contribution of outlier instances. A contrastive learning-assisted conditional alignment (CLCA) strategy is presented for distribution alignment. CLCA first exploits contrastive losses to discover the class discriminative information in both domains. It then employs a contrastive loss to match the clusters across the two domains based on adversarial domain learning. In this respect, CLCA attempts to reduce the domain discrepancy by matching the class-conditional and marginal distributions. Moreover, a new reweighting scheme is developed to improve the quality of weights estimation, which explores information from both the source and the target domains. Empirical results on several benchmark datasets demonstrate that the proposed CLA outperforms the existing state-of-the-art PDA methods.

Cross-scene wetland mapping on hyperspectral remote sensing images using adversarial domain adaptation network

Wetlands are one of the most important ecosystems on the Earth, and using hyperspectral remote sensing (RS) technology for fine wetland mapping is important for restoring and protecting the natural resources of coastal wetlands. However, the high cost in collecting labeled samples and inconsistent acquisition conditions across different geographic regions or scenes lead to difficulties in wetland mapping and classification. To mitigate these difficulties, a spatial–spectral weighted adversarial domain adaptation (SSWADA) network is proposed for the cross-scene wetland mapping using hyperspectral image (HSI). The proposed SSWADA employs an idea of weighted adversarial discrimination to align the feature distribution of source and target scenes, where a generator or feature extractor with joint 2D–3D convolution is used to extract spatial–spectral features of HSI, a weighted discriminator is constructed to perform source instance weighting and a multi-classifier structure is designed to improve the classification performance on target samples. Experimental results on four different tasks show that our SSWADA outperforms existing domain adaptation methods for cross-scene wetland mapping.

Multiple source partial knowledge transfer for manufacturing system modelling

Transfer learning has shown its attractiveness for manufacturing system modelling by leveraging previously acquired knowledge to assist in training the target model, whereas most techniques focus on single-source transfer settings. Since there are usually multiple source domains available in practice, multi-source transfer learning is attracting more attention. Existing researches regard the source instance or the source model as the basic information granularity, which makes it difficult to reduce the global shift and the local discrepancy across domains simultaneously. Therefore, this paper presents a multiple source partial knowledge transfer method (MSPKT) for manufacturing system modelling tasks, in which partial knowledge is defined as a novel information granularity between the instance granularity and model granularity. Firstly, TSK (Takagi-Sugeno-Kang) fuzzy system is introduced as the basic learner to represent partial knowledge effectively. Then, we design a transferability measurement of partial knowledge by considering the similarity and reliability to support transfer learning with multiple source domains. Finally, a synthetic dataset and two manufacturing system datasets are used to verify the effectiveness of the proposed method.

A transfer learning approach based on gradient boosting machine for diagnosis of Alzheimer's disease.

Early detection of Alzheimer’s disease (AD) during the Mild Cognitive Impairment (MCI) stage could enable effective intervention to slow down disease progression. Computer-aided diagnosis of AD relies on a sufficient amount of biomarker data. When this requirement is not fulfilled, transfer learning can be used to transfer knowledge from a source domain with more amount of labeled data than available in the desired target domain. In this study, an instance-based transfer learning framework is presented based on the gradient boosting machine (GBM). In GBM, a sequence of base learners is built, and each learner focuses on the errors (residuals) of the previous learner. In our transfer learning version of GBM (TrGB), a weighting mechanism based on the residuals of the base learners is defined for the source instances. Consequently, instances with different distribution than the target data will have a lower impact on the target learner. The proposed weighting scheme aims to transfer as much information as possible from the source domain while avoiding negative transfer. The target data in this study was obtained from the Mount Sinai dataset which is collected and processed in a collaborative 5-year project at the Mount Sinai Medical Center. The Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset was used as the source domain. The experimental results showed that the proposed TrGB algorithm could improve the classification accuracy by 1.5 and 4.5% for CN vs. MCI and multiclass classification, respectively, as compared to the conventional methods. Also, using the TrGB model and transferred knowledge from the CN vs. AD classification of the source domain, the average score of early MCI vs. late MCI classification improved by 5%.

A boosting-based transfer learning method to address absolute-rarity in skin lesion datasets and prevent weight-drift for melanoma detection

PurposeAutomated skin lesion analysis plays a vital role in early detection. Having relatively small-sized imbalanced skin lesion datasets impedes learning and dominates research in automated skin lesion analysis. The unavailability of adequate data poses difficulty in developing classification methods due to the skewed class distribution.Design/methodology/approachBoosting-based transfer learning (TL) paradigms like Transfer AdaBoost algorithm can compensate for such a lack of samples by taking advantage of auxiliary data. However, in such methods, beneficial source instances representing the target have a fast and stochastic weight convergence, which results in “weight-drift” that negates transfer. In this paper, a framework is designed utilizing the “Rare-Transfer” (RT), a boosting-based TL algorithm, that prevents “weight-drift” and simultaneously addresses absolute-rarity in skin lesion datasets. RT prevents the weights of source samples from quick convergence. It addresses absolute-rarity using an instance transfer approach incorporating the best-fit set of auxiliary examples, which improves balanced error minimization. It compensates for class unbalance and scarcity of training samples in absolute-rarity simultaneously for inducing balanced error optimization.FindingsPromising results are obtained utilizing the RT compared with state-of-the-art techniques on absolute-rare skin lesion datasets with an accuracy of 92.5%. Wilcoxon signed-rank test examines significant differences amid the proposed RT algorithm and conventional algorithms used in the experiment.Originality/valueExperimentation is performed on absolute-rare four skin lesion datasets, and the effectiveness of RT is assessed based on accuracy, sensitivity, specificity and area under curve. The performance is compared with an existing ensemble and boosting-based TL methods.

Integrating of Promising Computer Network Technology with Intelligent Supervised Machine Learning for Better Performance

The Software defined network (SDN) controller has such networks universal sight and allows for centralized management and control for the networks. The algorithms of Machine learning used alone or combined with the SDN controller's northbound applications in order to make intelligent SDN. SDN is such potential networking design that blends network's programmability with central administration. The control and the data planes are separated in SDN, and the network with central management point is called SDN controller, which may be programmed and utilized as a brain of the network. Lately, the community of researchers have shown a greater willingness to take advantage of current advances in artificial intelligence to give the SDN best decision making and learning skills. Our research found that combining SDN with Intelligent Supervised Machine Learning (ISML) is very important for performance improvement. ISML is the development of algorithms that can generate broad patterns and assumptions from external source instances in order to portend the predestination of future instances. The ISML algorithms of classification goal is to categorize data based on past information. In data science problems, classification is used rather frequently. To solve such problems, a number of successful approaches were already presented, including rule-based techniques, instance-based techniques, logic-based techniques, and stochastic techniques. This study examined the ISML algorithms' efficiency by checking the precision, accuracy, and with or without SDN recall.

Maximum Structural Generation Discrepancy for Unsupervised Domain Adaptation.

Unsupervised domain adaptation (UDA) has recently become an appealing research topic in visual recognition, since it exploits all accessible well-labeled source data to train a model with high generalization on target domain without any annotations. However, due to the significant domain discrepancy, the bottleneck for UDA is to learn effective domain-invariant feature representations. To fight off such an obstacle, we propose a novel cross-domain learning framework named Maximum Structural Generation Discrepancy (MSGD) to accurately estimate and mitigate domain shift via introducing an intermediate domain. First, the cross-domain topological structure is explored to propagate target samples to generate a novel intermediate domain paired with the specific source instances. The intermediate domain plays as the bridge to gradually reduce distribution divergence across source and target domains. Concretely, the similar category semantic across source and intermediate features tends to naturally conduct the class-level alignment on eliminating their domain shift. In terms of no target annotation, the domain-level alignment manner is suitable to narrow down the distance between intermediate and target domains. Moreover, to produce high-quality generative instances, we develop the class-driven collaborative translation (CDCT) module to generate class-consistent cross-domain samples in each mini-batch with the assistance of pseudo-labels. Extensive experimental analyses on five domain adaptation benchmarks demonstrate the effectiveness of our MSGD on solving UDA problem.

A Two-Way alignment approach for unsupervised multi-Source domain adaptation

Domain adaptation aims at transferring knowledge from labeled source domain to unlabeled target domain. Current advances primarily concern single source domain and neglect the setting of multiple source domains. Previous unsupervised multi-source domain adaptation (MDA) algorithms only consider domain-level alignment, while neglecting the category-level information among multiple domains and the instance variations inside each domain. This paper introduces a Two-Way alignment framework for MDA (TWMDA), which considers both domain-level and category-level alignments, and addresses the instance variations. We first align the target and multiple sources on the domain-level by an adversarial learning process. To circumvent the drawbacks of adversarial learning, we further reduce the domain gap on the category-level by minimizing the distance between the category prototypes and unlabeled target instances. To address the instance variations, we design an instance weighting strategy for diverse source instances. The effectiveness of TWMDA is demonstrated on three benchmark datasets for image classification.

Geoacoustic inversion for a 14-km autonomous underwater vehicle survey on the Malta Plateau

We consider signal processing and inversion of 1487 source instances recorded on a towed array along a 14-km seabed survey on the Malta Plateau. The data were acquired by autonomous underwater vehicle (AUV) and processed as reflection coefficients versus grazing angle and frequency. Data acquisition caused several artifacts that are studied by various data representations. These representations expose periodic fluctuations of reflection coefficients with angle and frequency that do not depend on seabed location. Averaging over source instances, frequencies, and angles reduces the artifacts. The Bayesian inversion assumes a one-dimensional seabed structure for each data set and is parametrized by an unknown number of homogeneous layers, sound velocities, densities, and attenuations. Results from individual inversions and sequential Monte Carlo sampling are compared. We demonstrate that removing data artifacts reduces over-fitting problems from previous considerations of the same data. Comparisons to piston and gravity core estimates, and separate wide-angle data show good agreement with the AUV results for two locations along the track. However, results at greater depths exhibit high uncertainty and strong influence of chosen prior boundaries. When considering results for all 1487 data sets, dipping and terminating layers are found along the track with high resolution (∼10 cm).

Eulerian time-marching in Vortex-In-Cell (VIC) method: reconstruction of multiple time-steps from a single vorticity volume and time-resolved boundary condition

A data assimilation approach is proposed to enhance the dynamic range of the Vortex-In-Cell (VIC) method by simulating future- and past- instances. The VIC method mainly considers a vorticity field from which velocity and acceleration fields are calculated through Poisson equations, respectively bounded by prescribed conditions. In addition, a vorticity time derivative is also available by the vorticity transport equation. The proposed approach focuses on such already available data, i.e., the vorticity and its time derivative fields, for simulating additional instances and getting feedbacks from the corresponding measurement instances, e.g., particle image velocimetry (PTV). However, the self-simulated flow field can be depleted due to a lack of incoming information, which is out of the reconstruction domain at the source instance. To supply that kind of information and thus sustain the simulation, boundary conditions of the simulated instances are required and considered. As a result, the proposed approach can gather corrections from multiple PTV instances while optimizing a single vorticity volume and time-resolved boundary conditions. Since the boundary grid points are much smaller in number than that of the whole volume, one can expect an increased dynamic range. A former work, VIC# (Jeon et al. 2018), which supplements additional constraints and coarse-grid approximation to VIC+ (Schneiders and Scarano 2016), is selected as a 3D method to which the proposed 4D approach is applied. Two explicit Eulerian time-marching methods are tested as a simulation scheme: the forward Euler and the Runge-Kutta methods. A numerical assessment is conducted using the synthetic PTV data, whose ground truth is known, and returns reconstruction qualities based on the velocity and the identified vortical structures. Other practical features regarding convergence and computation complexity are also reported. To visually verify an improvement by the proposed approach, two kinds of time-resolved Shake-the-Box (STB) measurements, which were acquired in high-speed systems, are processed and discussed.

Concept Drift-Tolerant Transfer Learning in Dynamic Environments.

Existing transfer learning methods that focus on problems in stationary environments are not usually applicable to dynamic environments, where concept drift may occur. To the best of our knowledge, the concept drift-tolerant transfer learning (CDTL), whose major challenge is the need to adapt the target model and knowledge of source domains to the changing environments, has yet to be well explored in the literature. This article, therefore, proposes a hybrid ensemble approach to deal with the CDTL problem provided that data in the target domain are generated in a streaming chunk-by-chunk manner from nonstationary environments. At each time step, a class-wise weighted ensemble is presented to adapt the model of target domains to new environments. It assigns a weight vector for each classifier generated from the previous data chunks to allow each class of the current data leveraging historical knowledge independently. Then, a domain-wise weighted ensemble is introduced to combine the source and target models to select useful knowledge of each domain. The source models are updated with the source instances performed by the proposed adaptive weighted CORrelation ALignment (AW-CORAL). AW-CORAL iteratively minimizes domain discrepancy meanwhile decreases the effect of unrelated source instances. In this way, positive knowledge of source domains can be potentially promoted while negative knowledge is reduced. Empirical studies on synthetic and real benchmark data sets demonstrate the effectiveness of the proposed algorithm.

Schema mapping coverage

In this paper, we introduce and study schema mapping coverage for relational databases. Given a relational schema mapping in the presence of both source dependencies and target dependencies, the coverage problem is to decide and describe which source instances have solutions under the mapping. Our main motivation is to describe limitations of schema mappings and hence to effectively determine if the mappings can fulfill their user expectations for the given tasks. We first propose using database dependencies to model user expectations of schema mappings. Then we formally define the notion of schema mapping coverage and propose using a set of dependencies in the source schema language to represent coverage. We look into how target dependencies indirectly enforce dependencies via mapping specifications on source instances and thus determine which source instances have solutions. We prove that the problem of computing schema mapping coverage is undecidable in general. We present algorithms for computing coverage for schema mappings where both the source dependencies and the target dependencies consist of functional dependencies and acyclic inclusion dependencies. Schema mapping coverage describes the ability to map source instances to target instances for mappings, it provides a way to describe limitations of schema mappings, which can be especially useful for mapping design and evolution.

FAST: Flexible and Low-Latency State Transfer in Mobile Edge Computing

Mobile Edge Computing (MEC) brings the benefits of cloud computing, such as computation, networking, and storage resources, close to end users, thus reducing end-to-end latency and enabling various novel use cases, such as vehicle platooning, autonomous driving, and the tactile internet. However, frequent user mobility makes it challenging for the MEC to guarantee the close proximity to the users. To tackle this challenge, the underlying network has to be capable of seamlessly migrating applications between multiple MEC sites. This application migration requires the quick and flexible migration of the application states without service interruption, while minimizing the state transfer cost. In this article, we first study the state transfer optimization problem in the MEC. To solve this problem, we propose a metaheuristic algorithm based on Tabu search. We then propose Flexible and Low-Latency State Transfer in Mobile Edge Computing (FAST), the first programmable state forwarding framework. FAST flexibly and directly forwards states between source instance and destination instance based on Software-Defined Networking (SDN). Both simulation results and practical testbed results demonstrate the favorable performance of the proposed Tabu search algorithm and the FAST framework compared to the state-of-the-art schemes.

Sequential Instance Refinement for Cross-Domain Object Detection in Images.

Cross-domain object detection in images has attracted increasing attention in the past few years, which aims at adapting the detection model learned from existing labeled images (source domain) to newly collected unlabeled ones (target domain). Existing methods usually deal with the cross-domain object detection problem through direct feature alignment between the source and target domains at the image level, the instance level (i.e., region proposals) or both. However, we have observed that directly aligning features of all object instances from the two domains often results in the problem of negative transfer, due to the existence of (1) outlier target instances that contain confusing objects not belonging to any category of the source domain and thus are hard to be captured by detectors and (2) low-relevance source instances that are considerably statistically different from target instances although their contained objects are from the same category. With this in mind, we propose a reinforcement learning based method, coined as sequential instance refinement, where two agents are learned to progressively refine both source and target instances by taking sequential actions to remove both outlier target instances and low-relevance source instances step by step. Extensive experiments on several benchmark datasets demonstrate the superior performance of our method over existing state-of-the-art baselines for cross-domain object detection.