Target Task Research Articles

Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-level Backdoor Attacks

The pre-training-then-fine-tuning paradigm has been widely used in deep learning. Due to the huge computation cost for pre-training, practitioners usually download pre-trained models from the Internet and fine-tune them on downstream datasets, while the downloaded models may suffer backdoor attacks. Different from previous attacks aiming at a target task, we show that a backdoored pre-trained model can behave maliciously in various downstream tasks without foreknowing task information. Attackers can restrict the output representations (the values of output neurons) of trigger-embedded samples to arbitrary predefined values through additional training, namely neuron-level backdoor attack (NeuBA). Since fine-tuning has little effect on model parameters, the fine-tuned model will retain the backdoor functionality and predict a specific label for the samples embedded with the same trigger. To provoke multiple labels in a specific task, attackers can introduce several triggers with predefined contrastive values. In the experiments of both natural language processing (NLP) and computer vision (CV), we show that NeuBA can well control the predictions for trigger-embedded instances with different trigger designs. Our findings sound a red alarm for the wide use of pre-trained models. Finally, we apply several defense methods to NeuBA and find that model pruning is a promising technique to resist NeuBA by omitting backdoored neurons.

Perceptual, procedural, and task learning for an auditory temporal discrimination task.

Perceptual learning reflects experience-driven improvements in the ability to detect changes in stimulus characteristics. The time course for perceptual learning overlaps with that for procedural learning (acquiring general skills and strategies) and task learning (learning the perceptual judgment specific to the task), making it difficult to isolate their individual effects. This study was conducted to examine the role of exposure to stimulus, procedure, and task information on learning for auditory temporal-interval discrimination. Eighty-three listeners completed five online sessions that required temporal-interval discrimination (target task). Before the initial session, listeners were differentially exposed to information about the target task's stimulus, procedure, or task characteristics. Learning occurred across sessions, but an exposure effect was not observed. Given the significant learning across sessions and variability within and across listeners, contributions from stimulus, procedure, and task exposure to overall learning cannot be discounted. These findings clarify the influence of experience on temporal perceptual learning and could inform designs of training paradigms that optimize perceptual improvements.

Semi-supervised Multi-task Learning with Auxiliary data

Compared with single-task learning, multi-tasks can obtain better classifiers by the information provided by each task. In the process of multi-task data collection, we always focus on the target task data in the training process, and ignore the non-target task data and unlabeled data that may be contained in the target task. In response to this issue, this paper introduces auxiliary or Universum into semi-supervised multi-task problem, and proposes a multi-task support vector machine (SU-MTLSVM) method based on semi-supervised learning to handle the case where each task contains the labeled, unlabeled, and Universum samples in the training set. This method introduces Universum as prior knowledge and provides high-dimensional information for semi-supervised learning, and builds a unique classifier from a large amount of unlabeled data. We then use KKT conditions and Lagrangian method to optimize the formulation of the model, and get the model parameters. Finally, we collect different data sets in the experiment part, and compare the performance of multiple baselines with the proposed method. Experiments prove that the method proposed in this paper is more effective for multi-task applications.

Meta-Learning Assisted Source Domain Optimization for Transfer Learning Based Optical Fiber Nonlinear Equalization

Transfer learning (TL) has been demonstrated its feasibility on fast remodeling for fiber nonlinearity equalization. It will be very efficient with fine-tuning rather than retraining when the channel parameters have been changed, e.g., the optical launch power and fiber distance. However, the performance is dependent on the highly correlation between the source and target tasks. An improper source domain cannot improve the efficiency or even negative transfer. In this paper, we propose a meta-learning assisted source domain optimization scheme in single- and multi-channel transmissions. We construct the support dataset and query dataset from the information of the inner-task and the across-task to train the neural network so that we can find a unified source domain and efficiently adapt to different tasks with fewer epochs. We conduct an experiment to transmit 120-Gb/s dual polarization (DP) 16-QAM signal over 800-km standard single mode fiber (SSMF). The required epochs for convergence can be significantly reduced and there is slight Q-factor improvement with meta-learning algorithm compared with retraining and conventional TL. We also analyze the TL strategy in multi-channel system. The simulation results show that meta-learning can also help to optimize the source domain with lower complexity.

Lateral Transfer Learning for Multiagent Reinforcement Learning

Some researchers have introduced transfer learning mechanisms to multiagent reinforcement learning (MARL). However, the existing works devoted to cross-task transfer for multiagent systems were designed just for homogeneous agents or similar domains. This work proposes an all-purpose cross-transfer method, called multiagent lateral transfer (MALT), assisting MARL with alleviating the training burden. We discuss several challenges in developing an all-purpose multiagent cross-task transfer learning method and provide a feasible way of reusing knowledge for MARL. In the developed method, we take features as the transfer object rather than policies or experiences, inspired by the progressive network. To achieve more efficient transfer, we assign pretrained policy networks for agents based on clustering, while an attention module is introduced to enhance the transfer framework. The proposed method has no strict requirements for the source task and target task. Compared with the existing works, our method can transfer knowledge among heterogeneous agents and also avoid negative transfer in the case of fully different tasks. As far as we know, this article is the first work denoted to all-purpose cross-task transfer for MARL. Several experiments in various scenarios have been conducted to compare the performance of the proposed method with baselines. The results demonstrate that the method is sufficiently flexible for most settings, including cooperative, competitive, homogeneous, and heterogeneous configurations.

A Meta-Learning Approach for Device-Free Indoor Localization

Although fingerprint-based methods could achieve high location accuracy for device-free indoor localization, it requires cost-expensive massive labeling. In order to fully exploit the previously collected CSI fingerprints for various indoor localization tasks, this letter proposes a meta-learning approach for indoor localization, with which an excellent initialization solution for a new localization task is resorted and further training with a few labeled samples results in sound performance improvement over the state-of-the-art deep learning (DL) approach. By recognizing the unequal contribution of an individual training task to the target task, a novel weighted loss is introduced under the framework of meta-learning. Experiments show that the proposed meta-learning approach can gain improvement of 20% Root-Mean-Square-Error (RMSE) enhancement over the DL approach.

Separating the effect of verbal cue on task-set activation into stimulus- and response-related processes: An eye-tracking study.

In task selection, a verbal cue is interpreted as more meaningful, and thus, it can elicit a faster response than an arbitrary cue. To investigate the effect of verbal cues on activating target task information, we combined an eye-tracking technique with a task-switching paradigm using an arbitrary cue and a type of verbal cue-a word cue with a short cue-target interval (CTI) and long CTI. We measured stimulus-selection time (time to orienting a stimulus) and postselection response time (time to respond to a stimulus after orienting to the stimulus) and separately examined the differential effect of cue types on these divided response times. Consequently, we found that word cues reduced stimulus-selection time and postselection response time compared with arbitrary sign cues in both the long and short CTI conditions. The results suggest that verbal cues activate task information more quickly, including a stimulus dimension and stimulus-response rule, than arbitrary cues. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Time-Frequency Feature-Based Seismic Response Prediction Neural Network Model for Building Structures

Currently, machine learning techniques are widely used in structural seismic response studies. The developed network models for various types of seismic response provide new ways to analyse seismic hazards. However, it is not easy to balance the applicability of the input, accuracy, and computational efficiency for existing network models. In this paper, a neural network model containing an efficient self-adaptive feature extraction module (AFEM) is designed. It can recognize time-frequency features from ground motion (GM) inputs for structural seismic response prediction tasks while considering the model’s computational accuracy and computational cost. The self-adaptive feature extraction module is constructed based on the MFCCs feature extraction process in NLP. AFEM recognizes time-frequency features closely related to structures’ behaviour and response under dynamic loads. Taking the seismic response prediction of a typical building as the target task, the neural network configuration, including a baseline model M0 and three comparison models (M1, M2, and M3) with AFEM, is systematically analysed. The results demonstrate that the proposed M1 model with initial AFEM, the M2 model with combined amplitude and phase features, and the M3 model with a complex-valued network are more adaptable than the baseline model to the target task. The extracted amplitude and phase features by the M3 model’s AFEM significantly improve model validation accuracy by 8.6% while reducing computation time by 11.4%. It could provide the basis for future research on regional earthquake damage intelligence assessment systems.

Linguodidactic foundations of linguistic provision of information security of an organization

The Department of Foreign Languages and Intercultural Communication at the Financial University under the Government of the Russian Federation is growing the status of an international platform for the research into Cognitive Linguistics and Professional Linguodidactics (PLD). It reflected the outcomes of that research both in the community-significant and large-scale conferences, and in the innovative educational process. The actualization of the inseparable connection between the research and the social and education practice is due to the opening of the undergraduate program in linguistics that was launched at the Financial University in the 2021–2022 academic year. The materials of this article reflect the first results of implementing the master’s program ‘Actual Problems of Linguodidactics and Methods of Teaching a Foreign Language. The core idea of this program rests in the definition of the linguodidactic foundations that reveal the structural and the content model of the development of the information and analytical competence of a future professional who would be capable of ensuring the information security of an organization by linguistic means. In the discussion part, the authors argue it proved the scientific and the practical interaction of the university teachers and their undergraduates to be embodied, on the one hand, in the course of the theoretical substantiation of the axiological strategies of foreign language education. (The latter determining the key concepts that play the role of the wording of the principles of PLD, i. e. the academic field that gained a new sound in the present research.) On the other hand, this interaction affects the selection of the adequate technologies that is necessitated by the said principles and the development of the specific target tasks that bring to life the mission the program’s focus, i. e. the linguistic support of the information security of an organization.

Transcranial direct current stimulation to the left dorsolateral prefrontal cortex enhances early dexterity skills with the left non-dominant hand: a randomized controlled trial

BackgroundThe left dorsolateral prefrontal cortex (DLPFC) is involved in early-phase manual dexterity skill acquisition when cognitive control processes, such as integration and complexity demands, are required. However, the effectiveness of left DLPFC transcranial direct current stimulation (tDCS) on early-phase motor learning and whether its effectiveness depends on the cognitive demand of the target task are unclear. This study aimed to investigate whether tDCS over the left DLPFC improves non-dominant hand dexterity performance and determine if its efficacy depends on the cognitive demand of the target task.MethodsIn this randomized, double-blind, sham-controlled trial, 70 healthy, right-handed, young adult participants were recruited. They were randomly allocated to the active tDCS (2 mA for 20 min) or sham groups and repeatedly performed the Purdue Pegboard Test (PPT) left-handed peg task and left-handed assembly task three times: pre-tDCS, during tDCS, and post tDCS.ResultsThe final sample comprised 66 healthy young adults (mean age, 22.73 ± 1.57 years). There were significant interactions between group and time in both PPT tasks, indicating significantly higher performance of those in the active tDCS group than those in the sham group post tDCS (p < 0.001). Moreover, a greater benefit was observed in the left-handed assembly task performance than in the peg task performance (p < 0.001). No significant correlation between baseline performance and benefits from tDCS was observed in either task.ConclusionsThese results demonstrated that prefrontal tDCS significantly improved early-phase manual dexterity skill acquisition, and its benefits were greater for the task with high cognitive demands. These findings contribute to a deeper understanding of the underlying neurophysiological mechanisms of the left DLPFC in the modulation of early-phase dexterity skill acquisition.Trial registration: This study was registered in the University Hospital Medical Information Network Clinical Trial Registry in Japan (UMIN000046868), Registered February 8, 2022 https://center6.umin.ac.jp/cgi-open-bin/ctr_e/ctr_view.cgi?recptno=R000053467

Class activation attention transfer neural networks for MCI conversion prediction

Accurate prediction of the trajectory of Alzheimer’s disease (AD) from an early stage is of substantial value for treatment and planning to delay the onset of AD. We propose a novel attention transfer method to train a 3D convolutional neural network to predict which patients with mild cognitive impairment (MCI) will progress to AD within 3 years. A model is first trained on a separate but related source task (task we are transferring information from) to automatically learn regions of interest (ROI) from a given image. Next we train a model to simultaneously classify progressive MCI (pMCI) and stable MCI (sMCI) (the target task we want to solve) and the ROIs learned from the source task. The predicted ROIs are then used to focus the model’s attention on certain areas of the brain when classifying pMCI versus sMCI. Thus, in contrast to traditional transfer learning, we transfer attention maps instead of transferring model weights from a source task to the target classification task. Our Method outperformed all methods tested including traditional transfer learning and methods that used expert knowledge to define ROI. Furthermore, the attention map transferred from the source task highlights known Alzheimer’s pathology.

Anomaly detection using a model-agnostic meta-learning-based variational auto-encoder for facility management

In a smart building, various types of sensor data are generated and employed to control devices within the building. Detecting anomalies in these sensor data is critical for effective facility management because it can prevent device malfunction or failure. Although deep learning-based methods have been used to detect anomalies, effective model construction is challenging because abnormal data tend to be rare under real-world conditions. In image classification and load forecasting, model-agnostic meta-learning (MAML) has recently been proposed to alleviate this problem using common knowledge learned from various tasks. This paper proposes an MAML-based unsupervised anomaly detection method called MAVAE for time-series sensor data. The proposed method uses a variational auto-encoder (VAE) as an anomaly detection model and adapts the model to a new target task with few unlabeled anomaly data via MAML. To our knowledge, this is the first study to train a VAE using MAML with time-series data. Extensive experiments on public data reveals that the proposed method outperforms existing anomaly detection methods, achieving an average improvement in prediction performance of 45% compared to state-of-the-art methods. The robustness of the proposed method is also demonstrated by evaluating its performance with a different number of samples for MAML. Our code is available at github.com/17011813/MAVAE.

Transfer Learning: A New Promising Techniques

Transfer Learning[1] is a machine learning technique that involves utilizing knowledge learned from one task to improve performance on another related task. This approach has been widely adopted in various fields such as computer vision, natural language processing, and speech recognition. The goal of this paper is to provide an overview of transfer learning and its recent developments. Transfer learning is particularly useful in situations where there is limited labeled data available for the target task. In these cases, the model can leverage knowledge learned from a related task with a larger amount of labeled data. This allows the model to overcome the problem of overfitting and improve performance on the target task.

A Novel Study on a Generalized Model Based on Self-Supervised Learning and Sparse Filtering for Intelligent Bearing Fault Diagnosis.

Recently, deep learning has become more and more extensive in the field of fault diagnosis. However, most deep learning methods rely on large amounts of labeled data to train the model, which leads to their poor generalized ability in the application of different scenarios. To overcome this deficiency, this paper proposes a novel generalized model based on self-supervised learning and sparse filtering (GSLSF). The proposed method includes two stages. Firstly (1), considering the representation of samples on fault and working condition information, designing self-supervised learning pretext tasks and pseudo-labels, and establishing a pre-trained model based on sparse filtering. Secondly (2), a knowledge transfer mechanism from the pre-training model to the target task is established, the fault features of the deep representation are extracted based on the sparse filtering model, and softmax regression is applied to distinguish the type of failure. This method can observably enhance the model's diagnostic performance and generalization ability with limited training data. The validity of the method is proved by the fault diagnosis results of two bearing datasets.

Self-taught cross-domain few-shot learning with weakly supervised object localization and task-decomposition

The shift between the source and target domains is the main challenge in cross-domain few-shot learning (CD-FSL) tasks. However, the target domain is unknown during training in the source domain, which results in a lack of directed guidance for target tasks. We observe that, owing to similar backgrounds in the target domains, self-labelled samples can be applied as prior tasks to transfer knowledge onto target tasks. Accordingly, we propose a task-expansion-decomposition framework for CD-FSL called the self-taught (ST) approach, which alleviates the problem of non-target guidance by constructing task-oriented metric spaces. Specifically, weakly supervised object localization (WSOL) and self-supervised technologies are employed to enrich task-oriented samples by exchanging and rotating discriminative regions, which generates a more abundant task set. Then, these tasks are decomposed into several tasks to complete few-shot recognition and rotation classification tasks. Transferring the source knowledge onto the target tasks and focusing on discriminative regions is beneficial to the task completion. We conducted extensive experiments under a cross-domain setting, including eight target domains: CUB, Cars, Places, Plantae, CropDieases, EuroSAT, ISIC, and ChestX. The experimental results demonstrate that the proposed ST approach is applicable to various metric-based models and provides promising improvements to CD-FSL.

A Wrapped Approach Using Unlabeled Data for Diabetic Retinopathy Diagnosis

Large-scale datasets, which have sufficient and identical quantities of data in each class, are the main factor in the success of deep-learning-based classification models for vision tasks. A shortage of sufficient data and interclass imbalanced data distribution, which often arise in the medical domain, cause modern deep neural networks to suffer greatly from imbalanced learning and overfitting. A diagnostic model of diabetic retinopathy (DR) that is trained from such a dataset using supervised learning is severely biased toward the majority class. To enhance the efficiency of imbalanced learning, the proposal of this study is to leverage retinal fundus images without human annotations by self-supervised or semi-supervised learning. The proposed approach to DR detection is to add an auxiliary procedure to the target task that identifies DR using supervised learning. The added process uses unlabeled data to pre-train the model that first learns features from data using self-supervised or semi-supervised learning, and then the pre-trained model is transferred with the learned parameter to the target model. This wrapper algorithm of learning from unlabeled data can help the model gain more information from samples in the minority class, thereby improving imbalanced learning to some extent. Comprehensive experiments demonstrate that the model trained with the proposed method outperformed the one trained with only the supervised learning baseline utilizing the same data, with an accuracy improvement of 4~5%. To further examine the method proposed in this study, a comparison is conducted, and our results show that the proposed method also performs much better than some state-of-the-art methods. In the case of EyePaCS, for example, the proposed method outperforms the customized CNN model by 9%. Through experiments, we further find that the models trained with a smaller but balanced dataset are not worse than those trained with a larger but imbalanced dataset. Therefore, our study reveals that utilizing unlabeled data can avoid the expensive cost of collecting and labeling large-scale medical datasets.

Orthogonal Transfer for Multitask Optimization

Knowledge transfer (KT) plays a key role in multitask optimization. However, most of the existing KT methods still face two challenges. First, the tasks may commonly have different dimensionalities (DDs), making the KT between heterogeneous search spaces very difficult. Second, the tasks may have different degrees of similarity in different dimensions, making that treating all dimensions with equal importance may be harmful to the KT process. To address these two challenges, this article proposes a novel orthogonal transfer (OT) method that is enabled by a cross-task mapping (CTM) strategy, which can achieve high-quality KT among heterogeneous tasks. For the first challenge, the CTM strategy maps the global best individual of one task from its original search space to the search space of the target task via an optimization process, which can handle the difference in task dimensionality. For the second challenge, the OT method is performed on the CTM-obtained individual and a random individual of the target task to find the best combination of different dimensions in these two individuals rather than treating all the dimensions equally, so as to achieve high-quality KT. To verify the effectiveness of the proposed OT method and the resulted OT-based multitask optimization (OTMTO) algorithm, this article not only uses the existing multitask optimization benchmark but also proposes a new benchmark test suite named multitask optimization problems (MTOPs) with DDs. Comprehensive experimental results on the existing and the proposed benchmarks show that the proposed OT method and the OTMTO algorithm are very advantageous in providing high-quality KT and in handling the heterogeneity of search space in MTOPs compared to the existing competitive evolutionary multitask optimization (EMTO) algorithms.

Aging effects on electrical and hemodynamic responses in the sensorimotor network during unilateral proximal upper limb functional tasks

Healthy aging leads to poorer performance in upper limb (UL) daily living movements. Understanding the neural correlates linked with UL functional movements may help to better understand how healthy aging affects motor control. Two non-invasive neuroimaging methods allow for monitoring the movement-related brain activity: functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG), respectively based on the hemodynamic response and electrical activity of brain regions. Coupled, they provide a better spatiotemporal mapping. The aim of this study was to evaluate the effect of healthy aging on the bilateral sensorimotor (SM1) activation patterns of functional proximal UL movements. Twenty-one young and 21 old healthy participants realized two unilateral proximal UL movements during: i) a paced reaching target task and ii) a circular steering task to capture the speed-accuracy trade-off. Combined fNIRS-EEG system was synchronised with movement capture system to record SM1 activation while moving. The circular steering task performance was significantly lower for the older group. The rate of increase in hemodynamic response was longer in the older group with no difference on the amplitude of fNIRS signal for the two tasks. The EEG results showed aging related reduction of the alpha-beta rhythms synchronisation but no desynchronisation modification. In conclusion, this study uncovers the age-related changes in brain electrical and hemodynamic response patterns in the bilateral sensorimotor network during two functional proximal UL movements using two complementary neuroimaging methods. This opens up the possibility to utilise combined fNIRS-EEG for monitoring the movement-related neuroplasticity in clinical practice.

Molecular Machine Learning for Chemical Catalysis: Prospects and Challenges.

ConspectusIn the domain of reaction development, one aims to obtain higher efficacies as measured in terms of yield and/or selectivities. During the empirical cycles, an admixture of outcomes from low to high yields/selectivities is expected. While it is not easy to identify all of the factors that might impact the reaction efficiency, complex and nonlinear dependence on the nature of reactants, catalysts, solvents, etc. is quite likely. Developmental stages of newer reactions would typically offer a few hundreds of samples with variations in participating molecules and/or reaction conditions. These "observations" and their "output" can be harnessed as valuable labeled data for developing molecular machine learning (ML) models. Once a robust ML model is built for a specific reaction under development, it can predict the reaction outcome for any new choice of substrates/catalyst in a few seconds/minutes and thus can expedite the identification of promising candidates for experimental validation. Recent years have witnessed impressive applications of ML in the molecular world, most of them aimed at predicting important chemical or biological properties. We believe that an integration of effective ML workflows can be made richly beneficial to reaction discovery.As with any new technology, direct adaptation of ML as used in well-developed domains, such as natural language processing (NLP) and image recognition, is unlikely to succeed in reaction discovery. Some of the challenges stem from ineffective featurization of the molecular space, unavailability of quality data and its distribution, in making the right choice of ML model and its technically robust deployment. It shall be noted that there is no universal ML model suitable for an inherently high-dimensional problem such as chemical reactions. Given these backgrounds, rendering ML tools conducive for reactions is an exciting as well as challenging endeavor at the same time. With the increased availability of efficient ML algorithms, we focused on tapping their potential for small-data reaction discovery (a few hundreds to thousands of samples).In this Account, we describe both feature engineering and feature learning approaches for molecular ML as applied to diverse reactions of high contemporary interest. Among these, catalytic asymmetric hydrogenation of imines/alkenes, β-C(sp3)-H bond functionalization, and relay Heck reaction employed a feature engineering approach using the quantum-chemically derived physical organic descriptors as the molecular features─all designed to predict the enantioselectivity. The selection of molecular features to customize it for a reaction of interest is described, along with emphasizing the chemical insights that could be gathered through the use of such features. Feature learning methods for predicting the yield of Buchwald-Hartwig cross-coupling, deoxyfluorination of alcohols, and enantioselectivity of N,S-acetal formation are found to offer excellent predictions. We propose a transfer learning protocol, wherein an ML model such as a language model is trained on a large number of molecules (105-106) and fine-tuned on a focused library of target task reactions, as an effective alternative for small-data reaction discovery (102-103 reactions). The exploitation of deep neural network latent space as a method for generative tasks to identify useful substrates for a reaction is demonstrated as a promising strategy.

The Effect of Task Fidelity on Learning Curves: A Synthetic Analysis

There have been discussions about the value of fidelity since simulation-based training systems have been created. A primary question, which has yet to be fully answered, is what is the effect of level of simulation fidelity on learning on a target task? We present a new analysis method and use it for several analyses of a training simulation for an electronic maintenance task with two levels of fidelity: a high-fidelity simulation that basically takes as much time as the real-world task and a low-fidelity simulation with minimal delays and many actions removed or reduced in fidelity and time. The analyses are based on the Keystroke-Level Model (KLM) and the power law of learning. The analyses predict that the performance on the low-fidelity simulation initially takes between one quarter and one eighth of the time of the high, and thus starts out providing between four and eight times as many practice trials in a given time period. The low-fidelity curve has a lower intercept and a steeper slope. Learners that move from low to high appear to not be adversely affected. For a small number of practice trials, this makes a significant difference. We also explore the effect of missing subtasks in the low-fidelity simulation. This effect varies with the tasks included: If the low-fidelity simulation does not train an important task, learners can be slower when they transfer. We also analyze a simulation that we have built and are studying. These analyses demonstrate that using lower fidelity training situations helps most where there is less time to practice, and that if there is extensive time to practice full fidelity has nearly the same outcome (but perhaps not the same costs or risks). We discuss how this analysis approach can help choose the level of fidelity of future training simulations.