Task Representation Research Articles

Semantic constraints to represent common sense required in household actions for multimodal learning-from-observation robot

The learning-from-observation (LfO) paradigm allows a robot to learn how to perform actions by observing human actions. Previous research in top-down learning-from-observation has mainly focused on the industrial domain, which consists only of the real physical constraints between a manipulated tool and the robot’s working environment. To extend this paradigm to the household domain, which consists of imaginary constraints derived from human common sense, we introduce the idea of semantic constraints, which are represented similarly to the physical constraints by defining an imaginary contact with an imaginary environment. By studying the transitions between contact states under physical and semantic constraints, we derive a necessary and sufficient set of task representations that provides the upper bound of the possible task set. We then apply the task representations to analyze various actions in top-rated household YouTube videos and real home cooking recordings, classify frequently occurring constraint patterns into physical, semantic, and multi-step task groups, and determine a subset that covers standard household actions. Finally, we design and implement task models, corresponding to these task representations in the subset, with the necessary daemon functions to collect the necessary parameters to perform the corresponding household actions. Our results provide promising directions for incorporating common sense into the robot teaching literature.

Memory representations in a cross-modal matching task: evidence for a verbal component

Memory representations in a cross-modal matching task: evidence for a verbal component

FedBnR: Mitigating federated learning Non-IID problem by breaking the skewed task and reconstructing representation

Federated Learning (FL), as a novel distributed machine learning paradigm, offers infinite possibilities for collaborative use of decentralized data among distributed entities. However, the potential data heterogeneity in distributed entities poses a great challenge to deploying FL for real-world practical applications. Inspired by the observed phenomenon of data heterogeneity simulation, we propose a FL framework to mitigate the non-iid problem by breaking the skewed task and reconstructing representation, called FedBnR. The basic idea of FedBnR is to break up the original skewed (unbalanced) task into multiple unskewed (balanced) subtasks and then reconstruct the representation of the original task using the unskewed subtask feature extractors. We design the FedBnR-SUR algorithm within the framework to verify its feasibility. We simulate the heterogeneous setup using the Dirichlet distribution. We conduct comparative experiments on MNIST, Cifar10, Cifar100, Cinic10, and Tiny-Imagenet. FedBnR-SUR outperforms the best baseline algorithms by about 0.96%, 1.49%, 3.53%, 1.77%, and 1.24%, respectively.

The emergence of task-relevant representations in a nonlinear decision-making task

This paper describes the relationship between performance in a decision-making task and the emergence of task-relevant representations. Participants learnt two tasks in which the appropriate response depended on multiple relevant stimuli and the underlying stimulus-outcome associations were governed by a latent feature that participants could discover. We divided participants into good and bad performers based on their overall classification rate and computed behavioural accuracy for each feature value. We found that participants with better performance had a better representation of the latent feature space. We then used representation similarity analysis on Electroencephalographic (EEG) data to identify when these representations emerge. We were able to decode task-relevant representations in a time window emerging 700 ms after stimulus presentation, but only for participants with good task performance. Our findings suggest that, in order to make good decisions, it is necessary to create and extract a low-dimensional representation of the task at hand.

Alteration of body representation in typical and atypical motor development.

Developmental coordination disorder (DCD) impacts the quality of life and ability to perform coordinated actions in 5% of school-aged children. The quality of body representations of individuals with DCD has been questioned, but never assessed. We hypothesize that children with DCD have imprecise body representations in the sensory and motor domains. Twenty neurotypical children, seventeen children with DCD (8-12 years old) and twenty neurotypical adults (25-45 years old) performed both sensory and motor body representation tasks: a limb identification and a limb movement task. We observed lower accuracy in the sensory task but not in the motor task. In both tasks, we observe a larger amplitude of errors, or synkinesis, in children with DCD than in neurotypical children. In neurotypical children, accuracy was lower than in neurotypical adults in the motor and sensory task, and the amplitude of sensory errors and synkinesis was higher than in neurotypical adults. Using a linear regression model, we showed that sensory accuracy is a good predictor of synkinesis production, and that synkinesis production is a good predictor of sensory accuracy, as can be expected by the perception-action loop. Results support the hypothesis of an imprecision of body representation in DCD. We suggest that this imprecision arises from noise in the body representation used at the level of internal models of action. Future studies may assess whether slower plasticity of body representations, initial imprecision, or both may account for this observation. At the clinical level, prevention strategies targeting body representation in early childhood are strategically important to limit such impairments. RESEARCH HIGHLIGHTS: Body representation is impaired in children with DCD and has a significant cost in terms of the accuracy of sensory identification of body parts and associated movements. Inaccuracies in the body representation measured in perception and in action (error amplitude and synkinesis) are related in both NT children and adults. In typical development, we provide evidence of a strong link between body schema and body image.

A Shapelet-Based Framework for Unsupervised Multivariate Time Series Representation Learning

Recent studies have shown great promise in unsupervised representation learning (URL) for multivariate time series, because URL has the capability in learning generalizable representation for many downstream tasks without using inaccessible labels. However, existing approaches usually adopt the models originally designed for other domains (e.g., computer vision) to encode the time series data and rely on strong assumptions to design learning objectives, which limits their ability to perform well. To deal with these problems, we propose a novel URL framework for multivariate time series by learning time-series-specific shapelet-based representation through a popular contrasting learning paradigm. To the best of our knowledge, this is the first work that explores the shapelet-based embedding in the unsupervised general-purpose representation learning. A unified shapelet-based encoder and a novel learning objective with multi-grained contrasting and multi-scale alignment are particularly designed to achieve our goal, and a data augmentation library is employed to improve the generalization. We conduct extensive experiments using tens of real-world datasets to assess the representation quality on many downstream tasks, including classification, clustering, and anomaly detection. The results demonstrate the superiority of our method against not only URL competitors, but also techniques specially designed for downstream tasks. Our code has been made publicly available at https://github.com/real2fish/CSL.

Transferable Adaptive Differential Evolution for Many-Task Optimization.

The evolutionary multitask optimization (EMTO) algorithm is a promising approach to solve many-task optimization problems (MaTOPs), in which similarity measurement and knowledge transfer (KT) are two key issues. Many existing EMTO algorithms estimate the similarity of population distribution to select a set of similar tasks and then perform KT by simply mixing individuals among the selected tasks. However, these methods may be less effective when the global optima of the tasks greatly differ from each other. Therefore, this article proposes to consider a new kind of similarity, namely, shift invariance, between tasks. The shift invariance is defined that the two tasks are similar after linear shift transformation on both the search space and the objective space. To identify and utilize the shift invariance between tasks, a two-stage transferable adaptive differential evolution (TRADE) algorithm is proposed. In the first evolution stage, a task representation strategy is proposed to represent each task by a vector that embeds the evolution information. Then, a task grouping strategy is proposed to group the similar (i.e., shift invariant) tasks into the same group while the dissimilar tasks into different groups. In the second evolution stage, a novel successful evolution experience transfer method is proposed to adaptively utilize the suitable parameters by transferring successful parameters among similar tasks within the same group. Comprehensive experiments are carried out on two representative MaTOP benchmarks with a total of 16 instances and a real-world application. The comparative results show that the proposed TRADE is superior to some state-of-the-art EMTO algorithms and single-task optimization algorithms.

Shared Task Representation for Human–Robot Collaborative Navigation: The Collaborative Search Case

Recent research in Human Robot Collaboration (HRC) has spread and specialised in many sub-fields. Many show considerable advances, but the human–robot collaborative navigation (HRCN) field seems to be stuck focusing on implicit collaboration settings, on hypothetical or simulated task allocation problems, on shared autonomy or on having the human as a manager. This work takes a step forward by presenting an end-to-end system capable of handling real-world human–robot collaborative navigation tasks. This system makes use of the Social Reward Sources model (SRS), a knowledge representation to simultaneously tackle task allocation and path planning, proposes a multi-agent Monte Carlo Tree Search (MCTS) planner for human–robot teams, presents the collaborative search as a testbed for HRCN and studies the usage of smartphones for communication in this setting. The detailed experiments prove the viability of the approach, explore collaboration roles adopted by the human–robot team and test the acceptability and utility of different communication interface designs.

Using Curiosity for an Even Representation of Tasks in Continual Offline Reinforcement Learning

Using Curiosity for an Even Representation of Tasks in Continual Offline Reinforcement Learning

Effects of mental rotation on map representation in orienteers-behavioral and fNIRS evidence.

Taking orienteering as an example, this study aimed to reveal the effects of mental rotation on orienteers' map representation and their brain processing characteristics. Functional near-infrared spectroscopic imaging (fNIRS) was used to explore the behavioral performance and cortical oxyhemoglobin concentration changes of map-represented cognitive processing in orienteering athletes under two task conditions: normal and rotational orientation. Compared to that in the normal orientation, athletes' task performance in the rotated orientation condition was significantly decreased, as evidenced by a decrease in correct rate and an increase in reaction time; in the normal orientation condition, blood oxygen activation in the dorsolateral prefrontal lobe was significantly greater than that in the ventral prefrontal lobe, which was significantly correlated with the correct rate. With rotating orientation, the brain oxygen average of each region of interest was enhanced, and the brain region specifically processed was the ventral prefrontal lobe, specifically correlating with the correct rate. Mental rotation constrains the map representation ability of athletes, and map representation in rotational orientation requires more functional brain activity for information processing. Ventral lateral prefrontal lobe activation plays an important role in the map representation task in rotational orientation.

BlendCSE: Blend contrastive learnings for sentence embeddings with rich semantics and transferability

Sentence representation is one of the most fundamental research topics in natural language processing (NLP), as its quality directly affects various downstream task performances. Recent studies for sentence representations have established state-of-the-art (SOTA) performance on semantic representation tasks. However, embeddings by those approaches share unsatisfying transferability when applied to various specific applications. Seldom work studies the transferability of semantic sentence embeddings. In this paper, we first explore the transferability characteristic of the sentence embeddings, and present BlendCSE, a new sentence embedding model targeting rich semantics and transferability. BlendCSE blends three recent advanced NLP learning methodologies, namely, continue learning on masked language modeling (MLM), contrastive learning (CL) with data augmentations (DA), and semantic supervised learning. The main objectives of BlendCSE are to capture token/word level information, diversified linguistic properties, and sentence semantics, respectively. Empirical studies demonstrate that BlendCSE captures semantics comparably well on STS tasks, yet surpasses existing methods on various transfer tasks, yielding even stronger transferability on document-level applications. Ablation studies verified that the three learning objectives synergy well to capture semantics and transferability effectively.

Focusing on features with higher importance degree: A significance-wise mechanism for image restoration

With the aim of retrieving high-quality images from corrupted versions, image restoration meets the extensive demand of application scenarios. State-of-the-art methods solve this problem by means of designing convolution blocks in multistage architectures. However, the existing methods blindly extract and fuse features without considering which parts of the features are more effective for image restoration. In this paper, we propose a significance-wise mechanism with the goal of identifying the importance degree of each region in the feature representation and providing crucial feature information. Through the calculation of nonlinear function in the significance-wise mechanism, the region which is more conducive to image restoration will get higher importance values. Two important types of information are generated within the attention mechanism, including: (i) feature-sufficient maps with abundant feature representations, and (ii) significance-wise maps with the importance degree of patch information in the corrupted images. With the coordination of feature-sufficient maps and significance-wise maps, the network architecture can focus higher attention on crucial parts of feature information. Furthermore, we design a multistage feature fusion block with the significance-wise mechanism. Compared with existing attention mechanisms, our significance-wise mechanism has the ability to identify and generate crucial feature representation for multiple image restoration tasks. Due to the novel design of the network, we successfully implement multiple restoration tasks only by fine-tuning the number of channels once on a network. Abundant quantitative and qualitative experiments demonstrate that the effective image restoration network (EIRN) outperforms existing state-of-the-art algorithms on eleven datasets across a series of image restoration tasks, including image deblurring, denoising, super-resolution, and deraining. The source code and pretrained models will be available at https://github.com/XinyueZhangqdu/EIRN.

Pre-Training Across Different Cities for Next POI Recommendation

The Point-of-Interest (POI) transition behaviors could hold absolute sparsity and relative sparsity very differently for different cities. Hence, it is intuitive to transfer knowledge across cities to alleviate those data sparsity and imbalance problems for next POI recommendation. Recently, pre-training over a large-scale dataset has achieved great success in many relevant fields, like computer vision and natural language processing. By devising various self-supervised objectives, pre-training models can produce more robust representations for downstream tasks. However, it is not trivial to directly adopt such existing pre-training techniques for next POI recommendation, due to the lacking of common semantic objects (users or items) across different cities . Thus in this paper, we tackle such a new research problem of pre-training across different cities for next POI recommendation. Specifically, to overcome the key challenge that different cities do not share any common object, we propose a novel pre-training model named CATUS , by transferring the cat egory-level u niversal tran s ition knowledge over different cities. Firstly, we build two self-supervised objectives in CATUS : next category prediction and next POI prediction , to obtain the universal transition-knowledge across different cities and POIs. Then, we design a category-transition oriented sampler on the data level and an implicit and explicit transfer strategy on the encoder level to enhance this transfer process. At the fine-tuning stage, we propose a distance oriented sampler to better align the POI representations into the local context of each city. Extensive experiments on two large datasets consisting of four cities demonstrate the superiority of our proposed CATUS over the state-of-the-art alternatives. The code and datasets are available at https://github.com/NLPWM-WHU/CATUS.

Investigation of the BERT model on nucleotide sequences with non-standard pre-training and evaluation of different k-mer embeddings.

In recent years, pre-training with the transformer architecture has gained significant attention. While this approach has led to notable performance improvements across a variety of downstream tasks, the underlying mechanisms by which pre-training models influence these tasks, particularly in the context of biological data, are not yet fully elucidated. In this study, focusing on the pre-training on nucleotide sequences, we decompose a pre-training model of Bidirectional Encoder Representations from Transformers (BERT) into its embedding and encoding modules to analyze what a pre-trained model learns from nucleotide sequences. Through a comparative study of non-standard pre-training at both the data and model levels, we find that a typical BERT model learns to capture overlapping-consistent k-mer embeddings for its token representation within its embedding module. Interestingly, using the k-mer embeddings pre-trained on random data can yield similar performance in downstream tasks, when compared with those using the k-mer embeddings pre-trained on real biological sequences. We further compare the learned k-mer embeddings with other established k-mer representations in downstream tasks of sequence-based functional prediction. Our experimental results demonstrate that the dense representation of k-mers learned from pre-training can be used as a viable alternative to one-hot encoding for representing nucleotide sequences. Furthermore, integrating the pre-trained k-mer embeddings with simpler models can achieve competitive performance in two typical downstream tasks. The source code and associated data can be accessed at https://github.com/yaozhong/bert_investigation.

Direct cortical inputs to hippocampal area CA1 transmit complementary signals for goal-directed navigation

The subregions of the entorhinal cortex (EC) are conventionally thought to compute dichotomous representations for spatial processing, with the medial EC (MEC) providing a global spatial map and the lateral EC (LEC) encoding specific sensory details of experience. Yet, little is known about the specific types of information EC transmits downstream to the hippocampus. Here, we exploit invivo sub-cellular imaging to record from EC axons in CA1 while mice perform navigational tasks in virtual reality (VR). We uncover distinct yet overlapping representations of task, location, and context in both MEC and LEC axons. MEC transmitted highly location- and context-specific codes; LEC inputs were biased by ongoing navigational goals. However, during tasks with reliable reward locations, the animals' position could be accurately decoded from either subregion. Our results revise the prevailing dogma about EC information processing, revealing novel ways spatial and non-spatial information is routed and combined upstream of the hippocampus.

Alibaba and forty thieves algorithm and novel Prioritized Prewitt Pattern(PPP)-based convolutional neural network (CNN) using hyperspherically compressed weights for facial emotion recognition

The visual representations created using the self-distillation paradigm of Bootstrap Your Emotion Latent (BYEL) are empirically found to be less evenly distributed than those created using proposed technique. This proposed work promotes the compression of weights on a hypersphere by minimizing the hyperspherical energy of network weights using a novel method of optimizing manifolds through Riemannian metrics and the Conjugate gradient technique. The proposed work demonstrates how regularising the networks of the BYEL architecture reduces the hyperspherical energy of neurons by directly optimising a measure of uniformity alongside the standard loss. This leads to more uniformly distributed representation and better performance for downstream tasks. The Alibaba and Forty Thieves Algorithm-based Optimization (AFTAO) methodology is used to select the most precise collection of hyperparameters for a novel Prioritized Prewitt Pattern (PPP)-based Convolutional Neural Network (CNN) that results in a higher accuracy for all the six datasets used for facial emotion recognition.

Experiences of Nurses Caring for Children Under Five Years with Burns at a Teaching Hospital in Ghana

Background Nurses provide comprehensive, person-centred care to patients of all ages with any condition to save and protect lives. Caring for children under 5 years with burns involves high-level skills, agility and stable emotions. This study sought to explore nurses’ experience when managing children under five years with burns at the burns unit of a teaching hospital in Ghana. Methods An exploratory, descriptive qualitative approach was used with a purposive sampling method to recruit 15 participants at saturation from the Burns Unit. The interviews were conducted using a semi-structured interview guide. With participants’ consent, their voices were audio-recorded. Data were transcribed verbatim after the researchers listened to the audios several times. Data were analysed using thematic content analysis. Results Narratives of the nurses were represented as emotional experiences, challenges and coping strategies associated with caring for children with burn injury. Mental representation of care task, grief and sadness, helplessness and impatience were identified as the emotional experiences. Increased workload, lack of adequate motivation and shortage of hospital supplies were the main challenges. Teamwork, good working relations, effective work-life balance and emotional detachment were the coping strategies the nurses adopted. Conclusion This study highlighted the experiences of nurses caring for children under five years with burns. The work environment of nurses created stress and increased the burden of care. Nurses need support to recognise and manage their physical, emotional and social responses to their work. In addition, periodic staff training must be promoted to boost staff confidence and enhance their efficiency on the job.

Sparse U-PDP: A Unified Multi-Task Framework for Panoptic Driving Perception

This paper proposes a multi-task unified decoding framework for the panoptic driving perception, Sparse U-PDP. This framework’s primary objective is to combine vehicle object detection, lane detection and drivable area segmentation tasks. This paper mainly builds a multi-task unified decoder and further explores whether the potential connection between multi-tasks can improve the robustness of the model. Experiments demonstrate that the proposed Sparse U-PDP outperforms the present state-of-the-art multi-task model in terms of accuracy. The primary contributions of this work are as follows: First, we present the approach to unified multi-task representation. We abstract the multi-task into the “dynamic convolution kernels” representation form to build a highly unified multi-task decoder. Second, we use the proposed dynamic interaction module to establish different feature sampling pipelines for various task features. Lastly, our model is verified on the BDD100K dataset, where we achieve an AP50 of 84.1 in the “vehicle” category, 32.0 IoU in the lane detection, and 93.0 mIoU in the drivable area segmentation with the helper of CSP-Darknet. That is to say; Sparse U-PDP verifies that a more unified task representation form can implicitly increase the mutual help between different task branches.

EXAMINATION OF DIAGRAMMATIC REPRESENTATION AND VERBAL PROBLEM-SOLVING REPRESENTATIONS OF PRIMARY SCHOOL STUDENTS

This study aimed to examine the diagrammatic representation skills and problem-solving performances of students according to their problem-solving representations. A cross-sectional survey design using quantitative methods was used in this study. The sample consisted of 31 second-grade and 41 third-grade students from a public primary school in Turkey. The Diagrammatic Representation Test and Mathematical Operations Test were used in this study. The data were analyzed with descriptive statistical analysis, the chi-square test, the independent samples t-test, discriminant analysis and logistic regression analysis. The findings indicated that while the preferred types of representations for solving verbal problems and problem-solving performance did not vary significantly based on grade level, scores obtained from the diagrammatic representation test exhibited significant differences. It was observed that students' problem-solving performance and diagrammatic skills could predict their preferred types of representations for solving verbal problems. Consequently, students who possess knowledge regarding effective representation preferences, as well as the ability to construct and utilize them, are more likely to generate appropriate and high-quality representations, leading to accurate problem-solving outcomes. This, in turn, enhances their performance in diagrammatic representation tasks.

Self-Supervised Convolutional Neural Network Learning in a Hybrid Approach Framework to Estimate Chlorophyll and Nitrogen Content of Maize from Hyperspectral Images

The new generation of available (i.e., PRISMA, ENMAP, DESIS) and future (i.e., ESA-CHIME, NASA-SBG) spaceborne hyperspectral missions provide unprecedented data for environmental and agricultural monitoring, such as crop trait assessment. This paper focuses on retrieving two crop traits, specifically Chlorophyll and Nitrogen content at the canopy level (CCC and CNC), starting from hyperspectral images acquired during the CHIME-RCS project, exploiting a self-supervised learning (SSL) technique. SSL is a machine learning paradigm that leverages unlabeled data to generate valuable representations for downstream tasks, bridging the gap between unsupervised and supervised learning. The proposed method comprises pre-training and fine-tuning procedures: in the first stage, a de-noising Convolutional Autoencoder is trained using pairs of noisy and clean CHIME-like images; the pre-trained Encoder network is utilized as-is or fine-tuned in the second stage. The paper demonstrates the applicability of this technique in hybrid approach methods that combine Radiative Transfer Modelling (RTM) and Machine Learning Regression Algorithm (MLRA) to set up a retrieval schema able to estimate crop traits from new generation space-born hyperspectral data. The results showcase excellent prediction accuracy for estimating CCC (R2 = 0.8318; RMSE = 0.2490) and CNC (R2 = 0.9186; RMSE = 0.7908) for maize crops from CHIME-like images without requiring further ground data calibration.