Target Task Data Research Articles

A local space transfer learning-based parallel Bayesian optimization with its application

The optimization of process parameters in polyolefin production can bring significant economic benefits to the factory. However, due to small data sets, high costs associated with parameter verification cycles, and difficulty in establishing an optimization model, the optimization process is often restricted. To address this issue, we propose using a transfer learning Bayesian optimization strategy to improve the efficiency of parameter optimization while minimizing resource consumption. Specifically, we leverage Gaussian process (GP) regression models to establish an integrated model that incorporates both source and target grade production task data. We then measure the similarity weights of each model by comparing their predicted trends, and utilize these weights to accelerate the solution of optimal process parameters for producing target polyolefin grades. In order to enhance the accuracy of our approach, we acknowledge that measuring similarity in a global search space may not effectively capture local similarity characteristics. Therefore, we propose a novel method for transfer learning optimization that operates within a local space (LSTL-PBO). This method employs partial data acquired through random sampling from the target task data and utilizes Bayesian optimization techniques for model establishment. By focusing on a local search space, we aim to better discern and leverage the inherent similarities between source tasks and the target task. Additionally, we incorporate a parallel concept into our method to address multiple local search spaces simultaneously. By doing so, we can explore different regions of the parameter space in parallel, thereby increasing the chances of finding optimal process parameters. This localized approach allows us to improve the precision and effectiveness of our optimization process. The performance of our method is validated through experiments on benchmark problems, and we discuss the sensitivity of its hyperparameters. The results show that our proposed method can significantly improve the efficiency of process parameter optimization, reduce the dependence on source tasks, and enhance the method's robustness. This has great potential for optimizing processes in industrial environments.

Semi-supervised Multi-task Learning with Auxiliary data

Compared with single-task learning, multi-tasks can obtain better classifiers by the information provided by each task. In the process of multi-task data collection, we always focus on the target task data in the training process, and ignore the non-target task data and unlabeled data that may be contained in the target task. In response to this issue, this paper introduces auxiliary or Universum into semi-supervised multi-task problem, and proposes a multi-task support vector machine (SU-MTLSVM) method based on semi-supervised learning to handle the case where each task contains the labeled, unlabeled, and Universum samples in the training set. This method introduces Universum as prior knowledge and provides high-dimensional information for semi-supervised learning, and builds a unique classifier from a large amount of unlabeled data. We then use KKT conditions and Lagrangian method to optimize the formulation of the model, and get the model parameters. Finally, we collect different data sets in the experiment part, and compare the performance of multiple baselines with the proposed method. Experiments prove that the method proposed in this paper is more effective for multi-task applications.

Facial Anatomical Landmark Detection Using Regularized Transfer Learning With Application to Fetal Alcohol Syndrome Recognition.

Fetal alcohol syndrome (FAS) caused by prenatal alcohol exposure can result in a series of cranio-facial anomalies, and behavioral and neurocognitive problems. Current diagnosis of FAS is typically done by identifying a set of facial characteristics, which are often obtained by manual examination. Anatomical landmark detection, which provides rich geometric information, is important to detect the presence of FAS associated facial anomalies. This imaging application is characterized by large variations in data appearance and limited availability of labeled data. Current deep learning-based heatmap regression methods designed for facial landmark detection in natural images assume availability of large datasets and are therefore not well-suited for this application. To address this restriction, we develop a new regularized transfer learning approach that exploits the knowledge of a network learned on large facial recognition datasets. In contrast to standard transfer learning which focuses on adjusting the pre-trained weights, the proposed learning approach regularizes the model behavior. It explicitly reuses the rich visual semantics of a domain-similar source model on the target task data as an additional supervisory signal for regularizing landmark detection optimization. Specifically, we develop four regularization constraints for the proposed transfer learning, including constraining the feature outputs from classification and intermediate layers, as well as matching activation attention maps in both spatial and channel levels. Experimental evaluation on a collected clinical imaging dataset demonstrate that the proposed approach can effectively improve model generalizability under limited training samples, and is advantageous to other approaches in the literature.

Source Model Selection for Deep Learning in the Time Series Domain

Transfer Learning aims to transfer knowledge from a source task to a target task. We focus on a situation when there is a large number of available source models, and we are interested in choosing a single source model that can maximize the predictive performance in the target domain. Existing methods compute some form of “similarity” between the source task data and the target task data. They then select the most similar source task and use the model trained on it for transfer learning. Previous methods do not account for the fact that it is the model parameters that are transferred rather than the data. Therefore, the “similarity” of the source data does not directly influence transfer learning performance. In addition, we would like the possibility of confidently selecting a source model even when the data it was trained on is not available, for example, due to privacy or copyright constraints. We propose to use the truncated source models as encoders for the target data. We then select a source model based on how well it clusters the target data in the latent encoding space, which we calculate using the Mean Silhouette Coefficient. We prove that if the encodings achieve a Mean Silhouette Coefficient of 1, optimal classification can be achieved using just the final layer of the target network. We evaluate our method using the University of California, Riverside (UCR) time series archive and show that the proposed method achieves comparable results to previous work, without using the source data.

A framework for semi-supervised metric transfer learning on manifolds

A common assumption of statistical learning theory is that the training and testing data are drawn from the same distribution. However, in many real-world applications, this assumption does not hold true. Hence, a realistic strategy, Cross Domain Adaptation (DA) or Transfer Learning (TA), can be used to employ previously labelled source domain data to boost the task in the new target domain. Previously, Cross Domain Adaptation methods have been focused on re-weighting the instances or aligning the cross-domain distributions. However, these methods have two significant challenges: (1) There is no proper consideration of the unlabelled data of target task as in the real-world, an abundant amount of unlabelled data is available, (2) The use of normal Euclidean distance function fails to capture the appropriate similarity or dissimilarity between samples. To deal with this issue, we have proposed a Semi-Supervised Metric Transfer Learning framework called SSMT that reduces the distribution between domains both statistically and geometrically by learning the instance weights, while a regularized distance metric is learned to minimize the within-class co-variance and maximize the between-class co-variance simultaneously for the target domain. Compared with the previous works where Mahalanobis distance metric and instance weights are learned by using the labelled data or in a pipelined framework that leads to a decrease in the performance, our proposed SSMT attempts to learn a regularized distance metric and instance weights by considering unlabelled data in a parallel framework. Experimental evaluation on three cross-domain visual data sets, e.g., PIE Face, Handwriting Digit Recognition on MNIST–USPS and Object Recognition, demonstrates the effectiveness of our designed approach on facilitating the unlabelled target task learning, compared to current state-of-the-art domain adaptation approaches.