Source Instances Research Articles

Domain Adversarial Reinforcement Learning for Partial Domain Adaptation.

Partial domain adaptation aims to transfer knowledge from a label-rich source domain to a label-scarce target domain (i.e., the target categories are a subset of the source ones), which relaxes the common assumption in traditional domain adaptation that the label space is fully shared across different domains. In this more general and practical scenario on partial domain adaptation, a major challenge is how to select source instances from the shared categories to ensure positive transfer for the target domain. To address this problem, we propose a domain adversarial reinforcement learning (DARL) framework to progressively select source instances to learn transferable features between domains by reducing the domain shift. Specifically, we employ a deep Q-learning to learn policies for an agent to make selection decisions by approximating the action-value function. Moreover, domain adversarial learning is introduced to learn a common feature subspace for the selected source instances and the target instances, and also to contribute to the reward calculation for the agent that is based on the relevance of the selected source instances with respect to the target domain. Extensive experiments on several benchmark data sets clearly demonstrate the superior performance of our proposed DARL over existing state-of-the-art methods for partial domain adaptation.

Transfer Learning for Anomaly Detection through Localized and Unsupervised Instance Selection

Anomaly detection attempts to identify instances that deviate from expected behavior. Constructing performant anomaly detectors on real-world problems often requires some labeled data, which can be difficult and costly to obtain. However, often one considers multiple, related anomaly detection tasks. Therefore, it may be possible to transfer labeled instances from a related anomaly detection task to the problem at hand. This paper proposes a novel transfer learning algorithm for anomaly detection that selects and transfers relevant labeled instances from a source anomaly detection task to a target one. Then, it classifies target instances using a novel semi-supervised nearest-neighbors technique that considers both unlabeled target and transferred, labeled source instances. The algorithm outperforms a multitude of state-of-the-art transfer learning methods and unsupervised anomaly detection methods on a large benchmark. Furthermore, it outperforms its rivals on a real-world task of detecting anomalous water usage in retail stores.

Instance-based ensemble deep transfer learning network: A new intelligent degradation recognition method and its application on ball screw

Degradation recognition plays an important role in improving the safety of mechanical operation. For most of the existing recognition methods, using large amounts of labeled training data that obey the same probability distribution as testing data is an important pre-requisite for training effective recognition model. Unfortunately, it is difficult to collectmassive labeled condition data for some machines like ball screw. This makes it a huge challenge to build reliable recognition model since the labeled training data collected in target domain are insufficient. However, large amounts of labeled data collected in different but related domain (called source domain) are usually available, which can be treated as auxiliary training data to help build better recognition model in target domain. Inspired by the idea of transferring knowledge from source domain to target domain, a new intelligent method named instance-based ensemble deep transfer learning network (IEDT) is proposed in this paper to recognize the degradation under various operating conditions. IEDT mainly consists of three parts: First, source instances that fit better with the target domain are selected by filtering out these unrelated samples iteratively. The remaining source instances are employed to assist insufficient labeled target data to train multiple stacked auto-encoders (SAEs) with different activation functions. And then, the trained SAEs are transferred to target domain as feature extractors. The extracted features are fed into support vector machine (SVM) to construct SAE-SVM models, which are further trained by limited target training data and used for degradation recognition. Finally, ensemble strategy is proposed to calculate final results based on multiple predicted labels of individual SAE-SVMs. Run-to-failure test of ball screw is carried out to collect experimental data under different working conditions. The effectiveness of the proposed IEDT method is validated by four transfer degradation recognition experiments of ball screw. Results indicate that the proposed IEDT method is superior to comparison methods in degradation recognition when there is only a small amount of labeled condition data.

Conditional Domain Adversarial Adaptation for Heterogeneous Defect Prediction

Heterogeneous defect prediction (HDP) has become a very active research field in software engineering, which predicts the maximum number of bug-suspiciousness modules of a target project by prediction models built on source project with heterogeneous metric set. At present, some researchers have proposed some HDP models with a promising performance. Most of existing HDP models adopted unsupervised transfer learning to map source project and target project into the same feature space, which only considered the metrics space, not the label information from source project and few part of target project. Meanwhile, the predictive ability of these HDP models in effort-aware context have not been compared. Therefore, we set up to investigate the effectiveness of label information on HDP, and to propose a HDP model for improving the predicting performance in classification and effort-aware contexts. In order to use these label information, we propose a novel conditional domain adversarial adaptation (CDAA) approach to tackle heterogeneous problem in SDP, which is motivated by generative adversarial networks (GANs). There are three networks in architecture of our CDAA, including one generator, one discriminator and one classifier. The generator learns how to transfer source instance space to target instance space. The discriminator learns how to identify the fake instances generated by generator. The classifier learns how to correctly classify the label of instances. In our CDAA, the loss function of classifier and discriminator are both back propagate to generator. Then, to ensure a fair comparison between state-of-the art methods and CDAA, we take AUC, MCC and $P_{opt}$ as measures to evaluate 28 open-source projects. Experimental results demonstrate that CDAA method could take advantage of label information to effectively map source project to target project and improve the predictive performance. Also, experimental results demonstrate that our CDAA method is not affected by the number of same metrics between source project and target project.

Multi-Source Selective Transfer Framework in Multi-Objective Optimization Problems

For complex system design [e.g., satellite layout optimization design (SLOD)] in practical engineering, when launching a new optimization instance with another parameter configuration from the intuition of designers, it is always executed from scratch which wastes much time to repeat the similar search process. Inspired by transfer learning which can reuse past experiences to solve relevant tasks, many researchers pay more attention to explore how to learn from past optimization instances to accelerate the target one. In real-world applications, there have been numerous similar source instances stored in the database. The primary question is how to measure the transferability from numerous sources to avoid the notorious negative transferring. To obtain the relatedness between source and target instance, we develop an optimization instance representation method named centroid distribution, which is by the aid of the probabilistic model learned by elite candidate solutions in estimation of distribution algorithm (EDA) during the evolutionary process. Wasserstein distance is employed to evaluate the similarity between the centroid distributions of different optimization instances, based on which, we present a novel framework called multisource selective transfer optimization with three strategies to select sources reasonably. To choose the suitable strategy, four selection suggestions are summarized according to the similarity between the source and target centroid distribution. The framework is beneficial to choose the most suitable sources, which could improve the search efficiency in solving multiobjective optimization problems. To evaluate the effectiveness of the proposed framework and selection suggestions, we conduct two experiments: 1) comprehensive empirical studies on complex multiobjective optimization problem benchmarks and 2) a real-world SLOD problem. Suggestions for strategy selection coincide with the experiment results, based on which, we propose a mixed strategy to deal with the negative transfer in the experiments successfully. The results demonstrate that our proposed framework achieves competitive performance on most of the benchmark problems in convergence speed and hypervolume values and performs best on the real-world applications among all the comparison algorithms.

TrAdaBoost Based on Improved Particle Swarm Optimization for Cross-Domain Scene Classification With Limited Samples

Scene classification is usually based on supervised learning, but collecting instances is expensive and time-consuming. TrAdaBoost has achieved great success in transferring source instances to target images, but it has problems, such as the excessive focus on instances harder to classify, the rapid convergence speed of the source instances, and the weight mismatch caused by the big gap between the number of source and target instances, leading to decreased classification accuracy. In this paper, in order to address these problems, classical particle swarm optimization (PSO) is modified to select the optimal feature subspace for classifying the “harder” and “easier” instances by reducing unimportant dimensions. A modified correction factor is proposed by considering the classification accuracy of the instances from both domains, to decrease the convergence speed. Iterative selective TrAdaBoost is also proposed to reduce the weight mismatch by removing the indiscriminate source instances. The experimental results obtained with three benchmark data sets confirm that the proposed method outperforms most of the previous methods of scene classification with limited target samples. It is also proved that modified PSO for optimal feature subspace selection, the modified correction factor, and iterative selective TrAdaBoost are all effective in improving the classification accuracy, giving improvements of 3.6%, 4.3%, and 2.7%, and these three contributions together increase the classification accuracy by about 8% in total.

Deep visual domain adaptation: A survey

Deep domain adaption has emerged as a new learning technique to address the lack of massive amounts of labeled data. Compared to conventional methods, which learn shared feature subspaces or reuse important source instances with shallow representations, deep domain adaption methods leverage deep networks to learn more transferable representations by embedding domain adaptation in the pipeline of deep learning. There have been comprehensive surveys for shallow domain adaption, but few timely reviews the emerging deep learning based methods. In this paper, we provide a comprehensive survey of deep domain adaptation methods for computer vision applications with four major contributions. First, we present a taxonomy of different deep domain adaption scenarios according to the properties of data that define how two domains are diverged. Second, we summarize deep domain adaption approaches into several categories based on training loss, and analyze and compare briefly the state-of-the-art methods under these categories. Third, we overview the computer vision applications that go beyond image classification, such as face recognition, semantic segmentation and object detection. Fourth, some potential deficiencies of current methods and several future directions are highlighted.

Transfer learning with partial related “instance-feature” knowledge

Abstract Transfer learning, which applies the knowledge from related but different source domains to improve the learning of the target domains, has attracted much attention in recent years. Lots of transfer learning methods have been proposed in literature, and many concentrate on selecting related source (-domain) instances or features for boosting learning. However, in each source instance, possibly only part of the features are related or helpful for transfer. That is, when a source instance is selected by relativeness, its unrelated feature knowledge can also be introduced. At the same time, some related feature knowledge may be discarded when an unrelated source instance is dropped. As a result, these methods ignore the partial related/unrelated knowledge in each source instance. In this paper, we attempt to discover such partial related “instance-feature” knowledge in transfer, and propose a new transfer learning method with partial related “instance-feature” knowledge (PRIF for short). Specifically, the partial “instance-feature” structure is first discovered by co-clustering over both instances and features, then the source instances are reconstructed considering both the related “instance-feature” knowledge and the related target (-domain) instances, so that the source instances can be more related to the target ones. Finally, transfer learning is performed with these newly-constructed source instances. Empirical results over several real-world datasets demonstrate the effectiveness of PRIF.

Limits of Schema Mappings

Schema mappings have been extensively studied in the context of data exchange and data integration, where they have turned out to be the right level of abstraction for formalizing data inter-operability tasks. Up to now and for the most part, schema mappings have been studied as static objects, in the sense that each time the focus has been on a single schema mapping of interest or, in the case of composition, on a pair of schema mappings of interest. In this paper, we adopt a dynamic viewpoint and embark on a study of sequences of schema mappings and of the limiting behavior of such sequences. To this effect, we first introduce a natural notion of distance on sets of finite target instances that expresses how “close” two sets of target instances are as regards the certain answers of conjunctive que- ries on these sets. Using this notion of distance, we investigate pointwise limits and uniform limits of sequences of schema mappings, as well as the companion notions of pointwise Cauchy and uniformly Cauchy sequences of schema mappings. We obtain a number of results about the limits of sequences of GAV schema mappings and the limits of sequences of LAV schema mappings that reveal striking differences between these two classes of schema mappings. We also consider the completion of the metric space of sets of target instances and obtain concrete representations of limits of sequences of schema mappings in terms of generalized schema mappings, that is, schema mappings with infinite target instances as solutions to (finite) source instances.

Conformal decision-tree approach to instance transfer

Instance transfer for classification aims at boosting generalization performance of classification models for a target domain by exploiting data from a relevant source domain. Most of the instance-transfer approaches assume that the source data is relevant to the target data for the complete set of features used to represent the data. This assumption fails if the target data and source data are relevant only for strict subsets of the input features which we call “partially input-feature relevant”. In this case these approaches may result in sub-optimal classification models or even in a negative transfer. This paper proposes a new decision-tree approach to instance transfer when the source data are partially input-feature relevant to the target data. The approach selects input features for tree nodes using univariate transfer of source instances. The instance transfer is guided by a conformal test for source relevance estimation. Experimental results on real-world data sets demonstrate that the new decision-tree approach is capable of outperforming existing instance-transfer approaches, especially, when the source data are partially input-feature relevant to the target data.

SEDEX: Scalable Entity Preserving Data Exchange

Data exchange is the process of generating an instance of a target schema from an instance of a source schema such that source data is reflected in the target. Generally, data exchnge is performed using schema mapping, representing high level relations between source and target schemas. In this paper, we argue that data exchange solely based on schema level information limits the ability to express semantics in data exchange. We show such schema level mappings not only may result in entity fragmentation, they are unable to resolve some ambiguous data exchange scenarios. To address this problem, we propose Scalable Entity Preserving Data Exchange (SEDEX), a hybrid method based on data and schema mapping that employs similarities between relation trees of source and target relations to find the best relations that can host source instances. Our experiments show SEDEX outperforms other methods in terms of quality and scalability of data exchange.

Exchange-Repairs

In a data exchange setting with target constraints, it is often the case that a given source instance has no solutions. In such cases, the semantics of target queries trivialize. The aim of this paper is to introduce and explore a new framework that gives meaningful semantics in such cases by using the notion of exchange-repairs. Informally, an exchange-repair of a source instance is another source instance that differs minimally from the first, but has a solution. Exchange-repairs give rise to a natural notion of exchange-repair certain answers (XR-certain answers) for target queries. We show that for schema mappings specified by source-to-target GAV dependencies and target equality-generating dependencies (egds), the XR-certain answers of a target conjunctive query can be rewritten as the consistent answers (in the sense of standard database repairs) of a union of conjunctive queries over the source schema with respect to a set of egds over the source schema, making it possible to use a consistent query-answering system to compute XR-certain answers in data exchange. We then examine the general case of schema mappings specified by source-to-target GLAV constraints, a weakly acyclic set of target tgds and a set of target egds. The main result asserts that, for such settings, the XR-certain answers of conjunctive queries can be rewritten as the certain answers of a union of conjunctive queries with respect to the stable models of a disjunctive logic program over a suitable expansion of the source schema.

Learning schema mappings

A schema mapping is a high-level specification of the relationship between a source schema and a target schema. Recently, a line of research has emerged that aims at deriving schema mappings automatically or semi-automatically with the help of data examples, that is, pairs consisting of a source instance and a target instance that depict, in some precise sense, the intended behavior of the schema mapping. Several different uses of data examples for deriving, refining, or illustrating a schema mapping have already been proposed and studied. In this article, we use the lens of computational learning theory to systematically investigate the problem of obtaining algorithmically a schema mapping from data examples. Our aim is to leverage the rich body of work on learning theory in order to develop a framework for exploring the power and the limitations of the various algorithmic methods for obtaining schema mappings from data examples. We focus on GAV schema mappings, that is, schema mappings specified by GAV (Global-As-View) constraints. GAV constraints are the most basic and the most widely supported language for specifying schema mappings. We present an efficient algorithm for learning GAV schema mappings using Angluin's model of exact learning with membership and equivalence queries. This is optimal, since we show that neither membership queries nor equivalence queries suffice, unless the source schema consists of unary relations only. We also obtain results concerning the learnability of schema mappings in the context of Valiant's well-known PAC (Probably-Approximately-Correct) learning model, and concerning the learnability of restricted classes of GAV schema mappings. Finally, as a byproduct of our work, we show that there is no efficient algorithm for approximating the shortest GAV schema mapping fitting a given set of examples, unless the source schema consists of unary relations only.

Getting unique solution in data exchange

A schema mapping is a high-level specification in which the relationship between two database schemas is described. In data exchange, schema mappings are one-way mappings that describe which data can be brought from source data to target data. Therefore, given a source instance and a mapping, there might be more than one valid target instance. This fact causes many problems in query answering over target data for non-conjunctive queries. To make query answering feasible for all queries, we focus on a methodology for extending the original schema mapping to guarantee the uniqueness of target instance corresponding to a source instance. To this end, we introduce a theoretical framework where the problem is transformed to an abduction problem, namely, definability abduction. We apply the framework to relational data exchange setting and solve the problem by pointing out minimal solutions according to a specific semantic minimality criterion.

Data exchange beyond complete data

In the traditional data exchange setting, source instances are restricted to be complete in the sense that every fact is either true or false in these instances. Although natural for a typical database translation scenario, this restriction is gradually becoming an impediment to the development of a wide range of applications that need to exchange objects that admit several interpretations. In particular, we are motivated by two specific applications that go beyond the usual data exchange scenario: exchanging incomplete information and exchanging knowledge bases . In this article, we propose a general framework for data exchange that can deal with these two applications. More specifically, we address the problem of exchanging information given by representation systems , which are essentially finite descriptions of (possibly infinite) sets of complete instances. We make use of the classical semantics of mappings specified by sets of logical sentences to give a meaningful semantics to the notion of exchanging representatives, from which the standard notions of solution, space of solutions, and universal solution naturally arise. We also introduce the notion of strong representation system for a class of mappings , that resembles the concept of strong representation system for a query language. We show the robustness of our proposal by applying it to the two applications mentioned above: exchanging incomplete information and exchanging knowledge bases, which are both instantiations of the exchanging problem for representation systems. We study these two applications in detail, presenting results regarding expressiveness, query answering and complexity of computing solutions, and also algorithms to materialize solutions.

Collaborative data sharing via update exchange and provenance

Recent work [Ives et al. 2005] proposed a new class of systems for supporting data sharing among scientific and other collaborations: this new collaborative data sharing system connects heterogeneous logical peers using a network of schema mappings. Each peer has a locally controlled and edited database instance, but wants to incorporate related data from other peers as well. To achieve this, every peer's data and updates propagate along the mappings to the other peers. However, this operation, termed update exchange , is filtered by trust conditions —expressing what data and sources a peer judges to be authoritative—which may cause a peer to reject another's updates. In order to support such filtering, updates carry provenance information. This article develops methods for realizing such systems: we build upon techniques from data integration, data exchange, incremental view maintenance, and view update to propagate updates along mappings, both to derived and optionally to source instances. We incorporate a novel model for tracking data provenance, such that curators may filter updates based on trust conditions over this provenance. We implement our techniques in a layer above an off-the-shelf RDBMS, and we experimentally demonstrate the viability of these techniques in the Orchestra prototype system.

Solutions and query rewriting in data exchange

Data exchange is the problem of taking data structured under a source schema and creating an instance of a target schema. Given a source instance, there may be many solutions – target instances that satisfy the constraints of the data exchange problem. Previous work has identified two classes of desirable solutions: canonical universal solutions, and their cores. Query answering in data exchange amounts to rewriting a query over the target schema to another query that, over a materialized target instance, gives the result that is semantically consistent with the source (specifically, the “certain answers”). Basic questions are then: (1) how do these solutions compare in terms of query rewriting? and (2) how can we determine whether a query is rewritable over a particular solution?Our goal is to answer these questions. Our first main result is that, in terms of rewritability by relational algebra queries, the core is strictly less expressive than the canonical universal solution, which in turn is strictly less expressive than the source. To develop techniques for proving queries non-rewritable, we establish structural properties of solutions; in fact they are derived from the technical machinery developed in the rewritability proofs. Our second result is that both the canonical universal solution and the core preserve the local structure of the data, and that every target query rewritable over any of these solutions cannot distinguish tuples whose neighborhoods in the source are similar. This gives us a first simple tool for checking whether a query is non-rewritable over the canonical universal solution or over the core. We also show that these tools generalize to arbitrary transformations that preserve the local structure of the data, and investigate an alternative semantics of query answering in data exchange.

Instance transfer boosting for object tracking

In this paper, we propose an Instance Transfer Boosting (ITB) framework for object tracking. The proposed tracking framework tries to transfer prior knowledge from the first frame and frame t-2 regarded as source instance to frame t-1 approximately as target instance. Those instances build the online training classifier used in tracking-by-detection for frame t. The novel method presents the tracking task in the current frame from the knowledge transferred by the first frame and the previous two frames, resulting in a more robust tracker for distinguishing the object from the background. Experimental results on several public video sequences demonstrated promising performance of the proposed tracking framework in both tracking accuracy and stability.

Characterizing schema mappings via data examples

Schema mappings are high-level specifications that describe the relationship between two database schemas; they are considered to be the essential building blocks in data exchange and data integration, and have been the object of extensive research investigations. Since in real-life applications schema mappings can be quite complex, it is important to develop methods and tools for understanding, explaining, and refining schema mappings. A promising approach to this effect is to use “good” data examples that illustrate the schema mapping at hand. We develop a foundation for the systematic investigation of data examples and obtain a number of results on both the capabilities and the limitations of data examples in explaining and understanding schema mappings. We focus on schema mappings specified by source-to-target tuple generating dependencies (s-t tgds) and investigate the following problem: which classes of s-t tgds can be “uniquely characterized” by a finite set of data examples? Our investigation begins by considering finite sets of positive and negative examples, which are arguably the most natural choice of data examples. However, we show that they are not powerful enough to yield interesting unique characterizations. We then consider finite sets of universal examples, where a universal example is a pair consisting of a source instance and a universal solution for that source instance. We first show that unique characterizations via universal examples is, in a precise sense, equivalent to the existence of Armstrong bases (a relaxation of the classical notion of Armstrong databases). After this, we show that every schema mapping specified by LAV s-t tgds is uniquely characterized by a finite set of universal examples with respect to the class of LAV s-t tgds. Moreover, this positive result extends to the much broader classes of n -modular schema mappings, n a positive integer. Finally, we study the unique characterizability of GAV schema mappings. It turns out that some GAV schema mappings are uniquely characterizable by a finite set of universal examples with respect to the class of GAV s-t tgds, while others are not. By unveiling a tight connection with homomorphism dualities, we establish an effective, sound, and complete criterion for determining whether or not a GAV schema mapping is uniquely characterizable by a finite set of universal examples with respect to the class of GAV s-t tgds.

Protected Research Information Management Environment (PRIME) provides a secure open source data management option for clinical and scientific research

Meeting abstract The Biomedical Information Sciences Unit (BISU) of the UT-CTSI developed a secure, centralized information management system that is accessed through web applications. This system, called Slim-Prim [1,2], is mounted on an Oracle server housed within the UT-ITS and currently hosts three multi-site clinical trials and several institutional trials. In addition, Slim-Prim houses databases of clinical and laboratory information and provides a centralized location for several physician-generated surveys. In the majority of these cases users have approached the BISU and asked for assistance with their data management needs. Slim-Prim [1,2] and the BISU provide a cost effective mechanism for funded groups to handle their data management needs. However, we are also aware that many smaller groups are in desperate need of secure yet flexible database solutions; thus there is the need for a portable, secure and open source instance of Slim-Prim. To this end we have developed and released an open source instance of the Slim-Prim system under the Affero GPL v3 (http://www.gnu.org/) called “Protected Research Information Management Environment (PRIME)”. The PRIME instance is written in PHP, based on an objected-oriented programming (OOP) architecture, and JavaScript with inverted file indexing search algorithms to render the graphical user interface (GUI) (http://www.php.net/). PRIME has a user-friendly interface that is menu driven with help features that assist the user in navigating through the application. PRIME exploits the MySQL database system and allows powerful SQL queries across tasks through using fully normalized table structures (http://www. mysql.com/). It is a web-enabled application that runs on most popular browsers. It is provided to the clinical and translational science community as a free and easily customized database solution.