Target Task Research Articles

Fault Location in Distribution Network Based on RNN and Transfer Learning

Fault location in distribution networks is a major challenge that needs to be addressed in power distribution systems. Currently, fault location methods based on matrix algorithms, genetic algorithms, deep learning, and other algorithms have received wide attention from the industry. However, such methods have some drawbacks: (1) they require high accuracy in fault information uploads, which can lead to low fault tolerance; (2) they tend to converge early and get stuck in local optimal solutions; (3) they involve high computational complexity, leading to location delays. In this study, we propose a fault location framework based on recurrent neural network (RNN) and transfer learning. In this method, we first encode the information data collected from distribution terminals, and then use RNN to establish a nonlinear mapping relationship between fault features and fault location intervals, which effectively improves fault tolerance and reduces misjudgment issues. We then use transfer learning to load the pre-trained model onto the target task to address the problem of insufficient data for fault location in distribution networks. Experimental results show that after 15 rounds of training, our T-RNN model has achieved over 80% accuracy. Benefiting from Glorot weight initialization adopted after transfer learning, the model achieves good performance early on compared to the BP model, converges faster, and ultimately achieves a prediction accuracy of 96.5%.

RDProtoFusion: Refined discriminative prototype-based multi-task fusion for cross-domain few-shot learning

Few-shot learning and meta-learning have been widely studied for their ability to reduce the burden of data annotation. However, real-world applications often involve training and target tasks from different source and target domains, leading to poor generalization of existing methods. To address this issue, we propose a novel method called RDProtoFusion, which leverages refined discriminative class prototypes to achieve task augmentation and bridge the domain shift. Firstly, we refine naive prototypes with query samples to avoid computation bias caused by the limited support size. Secondly, we perform multi-task fusion through class-representative prototypes, resulting in flexible task-augmentation with low complexity that effectively alleviates overfitting. Thirdly, we propose a prototype contrastive loss to obtain robust and accurate prototypes, by enhancing the discrimination between class prototypes. The proposed prototype refinement, fusion and contrast can improve generalization of the model effectively. We conduct extensive experiments on multiple cross-domain few-shot learning benchmarks, demonstrating that RDProtoFusion achieves state-of-the-art performance. Overall, our proposed method shows great potential for addressing the challenge of domain shift in few-shot learning, offering a promising avenue for real-world applications.

Selective policy transfer in multi-agent systems with sparse interactions

Previously trained single-agent strategies, which are considerably easier to acquire than multi-agent strategies, are instructive for multi-agent reinforcement learning, especially when the interactions between agents are sparse. Traditional methods of knowledge transfer, from single-agent source tasks to multi-agent target tasks, typically rely on a pre-designed Markov-decision-process similarity function or metric function for evaluating the task similarity. In this study, we propose a selective policy transfer (SPOT) algorithm that eliminates the need for the manual crafting of a metric function to assess the similarity between source and target tasks. The SPOT algorithm enables agents to autonomously determine when and which policy to transfer using well-trained single-agent policies as options in the training process. We introduced a multi-agent policy-learning option in the option library, thus allowing the SPOT algorithm to leverage the transferred knowledge while concurrently learning new policies. The SPOT algorithm efficiently transfers sequential strategies in a few steps, thereby capturing high-level semantics. Experimental results obtained in both multi-agent arcade and multi-agent particle environments have demonstrated that the proposed algorithm outperforms the state-of-the-art methods in terms of jump-start and convergence speeds. Our evaluation indicators included the jump start, steps to threshold, cumulative reward, and asymptotic performance. Furthermore, visualizations of the strategies and termination functions of the agents aid in elucidating the operational principles of the SPOT algorithm.

Immersive e-learning application in intelligent teaching of English composition based on neural network algorithm

Technical solutions for evaluating traditional English teaching composition and other problems, such as excessive subjectivity, waste of time, slow feedback, short scoring time, heavy tasks and so on. This paper first analyzes the theoretical basis of neural network algorithm, and completes the preprocessing of the composition text submitted by students from three aspects: text segmentation, text representation and text fragment retrieval. Then, an intelligent evaluation system of English composition is established, and the scoring standard of English composition is determined. On this basis, the design of English scoring module is improved, and an anti cheating module for common plagiarism problems in English composition is added. Finally, the English composition intelligent evaluation system is tested. It can effectively complete the target task, evaluate and correct the composition, and has a higher recognition success rate and accuracy rate. By studying the algorithm and applying it to the construction of English composition intelligent evaluation system, this paper successfully designs a kind of English composition intelligent evaluation system, which promotes the development of teaching work.

Self-paced method for transfer partial label learning

In partial label learning (PLL) problem, each training sample corresponds to a group of candidate labels, in which only one label is the ground-truth label (correct label). Almost all the existing PLL algorithms attempt to eliminate the ambiguity of the candidate label sets by treating all labels indiscriminately. However, this kind of approach lacks the consideration of the complexity between the labels and the instances in the training process. Encouraged by the extensive researches of the self-paced learning (SPL) and transfer learning (TL) in various fields, this paper introduces SPL and TL together to address the PLL problem and proposes a new SPL framework, which is called self-paced method for transfer partial label learning (SPTPLL). The proposed model utilizes transfer learning model to share the parameters and regularization terms of the Support Vector Machine (SVM), which can transfer knowledge from the source task to the target task. Additionally, we implement the self-paced learning scheme by choosing a suitable self-paced function to enhance the robustness of the proposed model. In the process of learning iteration, the priority of training examples with their candidate labels is ranked through self-paced learning to control the learning process. Finally, we demonstrate the superior performance of the proposed method through a large number of experiments compared with state-of-the-art baseline methods.

IfCMD: A Novel Method for Radar Target Detection under Complex Clutter Backgrounds

Traditional radar target detectors, which are model-driven, often suffer remarkable performance degradation in complex clutter environments due to the weakness in modeling the unpredictable clutter. Deep learning (DL) methods, which are data-driven, have been introduced into the field of radar target detection (RTD) since their intrinsic non-linear feature extraction ability can enhance the separability between targets and the clutter. However, existing DL-based detectors are unattractive since they require a large amount of independent and identically distributed (i.i.d.) training samples of target tasks and fail to be generalized to the other new tasks. Given this issue, incorporating the strategy of meta-learning, we reformulate the RTD task as a few-shot classification problem and develop the Inter-frame Contrastive Learning-Based Meta Detector (IfCMD) to generalize to the new task efficiently with only a few samples. Moreover, to further separate targets from the clutter, we equip our model with Siamese architecture and introduce the supervised contrastive loss into the proposed model to explore hard negative samples, which have the targets overwhelmed by the clutter in the Doppler domain. Experimental results on simulated data demonstrate competitive detection performance for moving targets and superior generalization ability for new tasks of the proposed method.

A Hybrid Preference Interaction Mechanism for Multi-Satellite Imaging Dynamic Mission Planning

The existing multi-satellite dynamic mission planning system hardly satisfies the requirements of fast response time and high mission benefit in highly dynamic situations. In the meantime, a reasonable decision-maker preference mechanism is an additional challenge for multi-satellite imaging dynamic mission planning based on user preferences (MSDMPUP). Therefore, this study proposes the hybrid preference interaction mechanism and knowledge transfer strategy for the multi-objective evolutionary algorithm (HPIM–KTSMOEA). Firstly, an MSDMPUP model based on a task rolling window is constructed to achieve timely updating of the target task importance degree through the simultaneous application of periodic triggering and event triggering methods. Secondly, the hybrid preference interaction mechanism is constructed to plan according to the satellite controller’s preference-based commands in different phases of the optimal search of the mission planning scheme to effectively respond to the dynamic changes in the environment. Finally, a knowledge transfer strategy for the multi-objective evolutionary algorithm is proposed to accelerate population convergence in new environments based on knowledge transfer according to environmental variability. Simulation experiments verify the effectiveness and stability of the method in processing MSDMPUP. This study found that the HPIM–KTSMOEA algorithm has high task benefit, short response time, and high task completion when processing MSDMPUP.

Cross-modal collaborative feature representation via Transformer-based multimodal mixers for RGB-T crowd counting

RGB-T crowd counting aims to utilize the thermal source data to compensate for the insufficient RGB feature representation under low-illumination conditions. However, how to efficiently use complementary information from different modalities is still a challenging issue for the target task. Existing CNN-based RGB-T crowd counting methods focus on the cross-modal feature representation via the local fixed-receptive-field convolutional operations and the transformer-based works mainly utilize the mutual attention, failing to realize the global cross-modal feature representation. To this end, this paper designs a multimodal counting network based on the multimodal transformer mixer to realize cross-modal collaborative feature representation. Firstly, we design a Transformer-based multimodal mixer to fully fuse the features from both modalities. And then we utilize the Transformer-based network with the multi-head self-attention layer replaced by the average pooling layer to build the two-stream backbone to extract rich multi-scale feature information of the two modalities. Meanwhile, the features extracted from the mixers are used to enhance the intermediate features to achieve the preservation of crowd detail information and suppression of background information. Finally, we design a pyramid regression head to aggregate the multi-scale feature maps to regress the density maps. Extensive experiments, as well as ablation studies, are conducted on challenging RGB-T crowd counting benchmark datasets (RGBT-CC and DroneRGBT), and the achieved competitive experimental results demonstrate the effectiveness of the proposed method on cross-modal collaborative feature representation for target task.

An integrated graph data privacy attack framework based on graph neural networks in IoT

SummaryKnowledge graphs contain a large amount of entity and relational data, and graph neural networks, as a class of efficient graph representation techniques based on deep learning, excel in knowledge graph modeling. However, previous neural network architectures for the most part only learn node representations and do not fully consider the heterogeneity of data. In this article, we innovatively propose a privacy attack framework based on IoT, PAFI, which is able to classify entities and relations, learn embedding representations in multi‐relational graphs, and can be applied to some existing neural network algorithms. Based on this, a fine‐grained privacy attack model, FPM, is proposed, which can perform attack operations on multiple targets, achieve selectivity of target tasks, and greatly improve the generalization ability of the attack model. In this article, the effectiveness of PAFI and FPM is demonstrated by real network datasets, and compared with previous attack methods, both of which achieve good results.

A deep neural network and transfer learning combined method for cross-task classification of error-related potentials.

Error-related potentials (ErrPs) are electrophysiological responses that naturally occur when humans perceive wrongdoing or encounter unexpected events. It offers a distinctive means of comprehending the error-processing mechanisms within the brain. A method for detecting ErrPs with high accuracy holds significant importance for various ErrPs-based applications, such as human-in-the-loop Brain-Computer Interface (BCI) systems. Nevertheless, current methods fail to fulfill the generalization requirements for detecting such ErrPs due to the high non-stationarity of EEG signals across different tasks and the limited availability of ErrPs datasets. This study introduces a deep learning-based model that integrates convolutional layers and transformer encoders for the classification of ErrPs. Subsequently, a model training strategy, grounded in transfer learning, is proposed for the effective training of the model. The datasets utilized in this study are available for download from the publicly accessible databases. In cross-task classification, an average accuracy of about 78% was achieved, exceeding the baseline. Furthermore, in the leave-one-subject-out, within-session, and cross-session classification scenarios, the proposed model outperformed the existing techniques with an average accuracy of 71.81, 78.74, and 77.01%, respectively. Our approach contributes to mitigating the challenge posed by limited datasets in the ErrPs field, achieving this by reducing the requirement for extensive training data for specific target tasks. This may serve as inspiration for future studies that concentrate on ErrPs and their applications.

UAS Visual Navigation in Large and Unseen Environments via a Meta Agent

Abstract. The aim of this work is to develop an approach that enables Unmanned Aerial System (UAS) to efficiently learn to navigate in large-scale urban environments and transfer their acquired expertise to novel environments. To achieve this, we propose a metacurriculum training scheme. First, meta-training allows the agent to learn a master policy to generalize across tasks. The resulting model is then fine-tuned on the downstream tasks. We organize the training curriculum in a hierarchical manner such that the agent is guided from coarse to fine towards the target task. In addition, we introduce Incremental Self-Adaptive Reinforcement learning (ISAR), an algorithm that combines the ideas of incremental learning and meta-reinforcement learning (MRL). In contrast to traditional reinforcement learning (RL), which focuses on acquiring a policy for a specific task, MRL aims to learn a policy with fast transfer ability to novel tasks. However, the MRL training process is time consuming, whereas our proposed ISAR algorithm achieves faster convergence than the conventional MRL algorithm. We evaluate the proposed methodologies in simulated environments and demonstrate that using this training philosophy in conjunction with the ISAR algorithm significantly improves the convergence speed for navigation in large-scale cities and the adaptation proficiency in novel environments. The project page is publicly available at https://superhan2611.github.io/isar_nav/.

Learning general features to bridge the cross-domain gaps in few-shot learning

Few-shot learning (FSL) methods have attracted great attention in recent years. However, under the cross-domain few-shot learning (CDFSL) setting, where domain gaps exist between the source and target domains, many FSL methods face performance degradation. One possible reason is the well-trained model can easily learn discriminative features on the source domain but may fail on target domains. To alleviate this problem, it is desirable to learn a model that can extract useful general features from target tasks regardless of domain gaps during the source domain training. In this paper, we propose a dynamic representation enhancement (DRE) framework that maximizes the model’s representation ability to learn diverse and meaningful features on unknown domains. First, we proposed a stage-wise cycling and warm-up feature activation training strategy to make the model continuously explore useful general features during training, instead of just satisfying with fitting well on the limited training data. Second, we designed a task-adaptive loss function to enhance the learning of diverse features by making the model focus on inter-class discriminative features during training. Experimental analysis shows that the proposed DRE framework effectively extracts informative features across various target domains. On the popular cross-domain BSCD-FSL benchmark, the performance of the DRE framework is competitive with the state-of-the-art methods with an average classification accuracy of 51.01%, 66.71% and 74.13% in the 5-way 1-shot, 5-way 5-shot and 5-way 20-shot settings, respectively. Our code is publicly available at https://github.com/ideal-123/Dynamic-Representation-Enhancement-framework.

Diverse and tailored image generation for zero-shot multi-label classification

Recently, zero-shot multi-label classification has garnered considerable attention owing to its capacity to predict unseen labels without human annotations. Nevertheless, prevailing approaches often use seen classes as imperfect proxies for unseen classes, resulting in suboptimal performance. Drawing inspiration from the success of text-to-image generation models in producing realistic images, we propose an innovative solution: generating synthetic data to construct a training set explicitly tailored for proxyless training of unseen labels. Our approach introduces a novel image generation framework that produces multi-label synthetic images of unseen classes for classifier training. To enhance the diversity of the generated images, we leverage a pretrained large language model to generate diverse prompts. Employing a pretrained multimodal contrastive language-image pretraining (CLIP) model as a discriminator, we assessed whether the generated images accurately represented the target classes. This enables automatic filtering of inaccurately generated images and preserves classifier accuracy. To refine the text prompts for more precise and effective multi-label object generation, we introduced a CLIP score-based discriminative loss to fine-tune the text encoder in the diffusion model. In addition, to enhance the visual features of the target task while maintaining the generalization of the original features and mitigating catastrophic forgetting resulting from fine-tuning the entire visual encoder, we propose a feature fusion module inspired by transformer attention mechanisms. This module aids in capturing the global dependencies between multiple objects more effectively. Extensive experimental results validated the effectiveness of our approach, demonstrating significant improvements over state-of-the-art methods. The code is available at https://github.com/TAKELAMAG/Diff-ZS-MLC.

Age-related changes in brain oscillatory patterns during an n-back task in children and adolescents

The development of brain oscillatory responses and their possible role in the working memory (WM) performance of children, adolescents and young adults was investigated. A set of 0- and 1-back tasks with letter stimuli were administered to a final sample of 131 subjects (between 6 and 20 years of age). A decrease in response times (RTs) and an increase of the sensitivity index d-prime (d′) were seen with increased age. RTs increased and d′ decreased with load, indicating higher difficulty for higher loads. Event-related synchronization (ERS) and event-related desynchronization (ERD) were obtained by the convolution of Morlet wavelets on the recorded EEG. Statistical analyses were performed of the absolute and relative power of brain oscillations defined by topography, frequency and latency. Posterior alpha and beta ERD, and frontocentral theta ERS, were induced by the stimuli presented during the n-back task. While relative theta ERS increased with age, absolute theta ERS, absolute and relative alpha and, absolute beta ERD, decreased with age. Age-related improvement in behavioral performance was mediated by relative theta. Alpha and beta ERD were more pronounced for the most difficult task (1-back) and for the target condition. Globally, there was high consistency of the effects of target type and task load across development. Theta ERS maturation is a crucial step for improving WM performance during development, while alpha and beta ERD maturation seem to be less critical for behavioral performance improvement with age, possibly due to a sufficient level of alpha-beta ERD for good performance in young children.

DAPM-CDR: A domain adaptation prompting model for drug response prediction

The rising incidence and mortality rates of cancer present significant challenges to global health. Variations in tumor growth rates and treatment responses have revealed the limitations of traditional therapies, highlighting the urgent need for predictive models for cancer drug responses based on computational methods. Current drug response prediction methods often fall short in accurately predicting responses for rare cancers with limited data. To address these issues, we propose a domain adaptation prompting model for drug response prediction, DAPM-CDR. DAPM-CDR is trained with cancer types with rich response data of cancer cell lines, integrating both common and specific information into prompts through contrastive learning, to transfer knowledge to target tasks that with sparse data on cancer cell line responses, thereby enhancing the generalization capabilities across various cancer types. The experiment results demonstrate that DAPM-CDR outperforms several competitive methods in predicting drug responses of cell lines, particularly excelling with data from rare cancers and showing significant performance enhancements.

Neural architecture search via similarity adaptive guidance

Evolutionary neural network architecture search (ENAS) has attracted the attention of many experts due to its global optimization capabilities to automatically search for convolutional neural network architectures based on the target task. The current search space for ENAS is not to design a fully structured network, but to search for smaller cell architectures to reduce search costs. However, blind search strategies do not effectively utilize the potential experience of the population. In order to utilize the potential experience learned by the current population to guide the evolutionary search of the population, we propose a similarity guided neural network architecture search algorithm based on cell architecture, which utilizes the similarity between pairwise architectures in the population as empirical knowledge learned by the population. Our proposed algorithm provides a novel method for calculating architecture similarity, which calculates architecture similarity separately from the cell and macro-structure. Then we decouple the connections and operations in the cell and calculate connection and operation similarity separately. In addition, we propose adaptive similarity selection and binary tournament selection strategies to enhance the algorithm’s global and local search capabilities and effectively explore the search space. Finally, we design an improved single-point crossover operator to enhance the local search ability of the evolutionary operator. The experimental results show that SAGNAS is a competitive algorithm that achieves 97.44% and 81.60% in CIFAR10 and CIFAR100 with only 1.9 GPU-days spent.

Multi-Instance Multi-Task Learning for Joint Clinical Outcome and Genomic Profile Predictions From the Histopathological Images.

With the remarkable success of digital histopathology and the deep learning technology, many whole-slide pathological images (WSIs) based deep learning models are designed to help pathologists diagnose human cancers. Recently, rather than predicting categorical variables as in cancer diagnosis, several deep learning studies are also proposed to estimate the continuous variables such as the patients' survival or their transcriptional profile. However, most of the existing studies focus on conducting these predicting tasks separately, which overlooks the useful intrinsic correlation among them that can boost the prediction performance of each individual task. In addition, it is sill challenge to design the WSI-based deep learning models, since a WSI is with huge size but annotated with coarse label. In this study, we propose a general multi-instance multi-task learning framework (HistMIMT) for multi-purpose prediction from WSIs. Specifically, we firstly propose a novel multi-instance learning module (TMICS) considering both common and specific task information across different tasks to generate bag representation for each individual task. Then, a soft-mask based fusion module with channel attention (SFCA) is developed to leverage useful information from the related tasks to help improve the prediction performance on target task. We evaluate our method on three cancer cohorts derived from the Cancer Genome Atlas (TCGA). For each cohort, our multi-purpose prediction tasks range from cancer diagnosis, survival prediction and estimating the transcriptional profile of gene TP53. The experimental results demonstrated that HistMIMT can yield better outcome on all clinical prediction tasks than its competitors.

Selective Random Walk for Transfer Learning in Heterogeneous Label Spaces.

Transfer learning has been widely used in different scenarios, especially in those lacking enough labeled data. However, most of the existing transfer learning methods are based on the assumption that the source and target domains should share the label space entirely or partially, which greatly limits their application scopes. In this article, a Selective Random Walk (SRW) method for transfer learning in heterogeneous label spaces is proposed to make full use of unlabeled auxiliary data, which acts as a bridge for knowledge transfer from the source domain to the target domain. The proposed SRW method can explicitly identify transfer sequences between source and target instances via auxiliary instances based on random walk techniques. Since not all of the transfer sequences generated by random walk are credible for the target task, the SRW method can learn to weight transfer sequences adaptively. Based on the weights of the transfer sequences, the SRW method leverages knowledge by forcing adjacent data points in the transfer sequence to be similar and making the target data point in the sequence represented by other data points in the same sequence. Experiments show that the SRW method outperforms state-of-the-art models in plenty of transfer learning tasks with heterogeneous label spaces constructed within and across several benchmark datasets.

Priority influences task selection decisions in multi-task management

The role of task priority on task selection in multi-task management is unclear based on prior work, leading to a common finding of ‘priority neglect’. However, properties such as urgency and conflict may influence whether operators weigh priority in their decision. We examined the role of instructed task prioritization, bolstered by more urgent and conflicting conditions, on how operators select among emergent, concurrent tasks when multitasking. Using the Multi-Attribute Task Battery (MATB) multitasking platform we tested both an auditory communications task and a manual tracking task as the priority tasks. Results showed that instructed priority significantly increased target task selection under the conflicting task conditions for both tasks. Urgency itself may modulate whether instructions to prioritize affect task selection choices when multitasking, and therefore counter to prior results instructions may yet be useful for helping operators select a higher priority task under conflict, a generalizable effect to be further explored.

Self-supervised dynamic and static feature representation learning method for flotation monitoring

Froth flotation is a widely used beneficiation method in mineral processing. Accurate grade monitoring is crucial for efficient minerals separation. However, despite recent developments highlighting the potential of deep learning in flotation monitoring, these methods face limitations in application due to the lack of reliable feature extraction and the scarcity of large datasets related to target task. To address these issues, we propose a self-supervised dynamic and static feature collaborative representation learning method. It extracts the spatiotemporal information from froth videos based on a two-stream convolutional architecture and is trained on a video frame prediction task without labeled data until both appearance and motion constraints are satisfied. Experiments on industrial zinc flotation data demonstrate that the learned representations transfer well to downstream grade monitoring tasks. This approach not only solves the label shortage problem when training large-scale deep learning models but also outperforms the state-of-the-art methods in flotation monitoring.