Encoder-decoder Model Research Articles

APT Attack Detection Method Based on Traceability Graph

This article proposes an Advanced Persistent Threat (APT) attack detection method based on traceability graphs, aimed at addressing the complexity and concealment of APT attacks. This method describes system behavior by constructing a traceability graph, optimizing it to reduce redundant information, converting the traceability graph sequence into a feature vector sequence, and using an encoder decoder model to train the GRU (Gate Recurrent Unit) model to extract long-term features of the sequence. Finally, a normal behavior model is established through clustering to detect APT attacks.

A Survey of Visual Transformers.

Transformer, an attention-based encoder-decoder model, has already revolutionized the field of natural language processing (NLP). Inspired by such significant achievements, some pioneering works have recently been done on employing Transformer-liked architectures in the computer vision (CV) field, which have demonstrated their effectiveness on three fundamental CV tasks (classification, detection, and segmentation) as well as multiple sensory data stream (images, point clouds, and vision-language data). Because of their competitive modeling capabilities, the visual Transformers have achieved impressive performance improvements over multiple benchmarks as compared with modern convolution neural networks (CNNs). In this survey, we have reviewed over 100 of different visual Transformers comprehensively according to three fundamental CV tasks and different data stream types, where taxonomy is proposed to organize the representative methods according to their motivations, structures, and application scenarios. Because of their differences on training settings and dedicated vision tasks, we have also evaluated and compared all these existing visual Transformers under different configurations. Furthermore, we have revealed a series of essential but unexploited aspects that may empower such visual Transformers to stand out from numerous architectures, e.g., slack high-level semantic embeddings to bridge the gap between the visual Transformers and the sequential ones. Finally, two promising research directions are suggested for future investment. We will continue to update the latest articles and their released source codes at https://github.com/liuyang-ict/awesome-visual-transformers.

A deep learning based encoder-decoder model for speed planning of autonomous electric truck platoons

Electric truck platooning offers a promising solution to extend the range of electric vehicles during long-haul operations. However, optimizing the platoon speed to ensure efficient energy utilization remains a critical challenge. The existing research on implementing data-driven solutions for truck platooning remains limited and implementing first principles solution is still a challenge. However, recognizing the resemblance of truck platoon data to a time series serves as a compelling motivation to explore suitable analytical techniques to address the problem. This paper presents a novel deep learning approach using a sequence-to-sequence encoder-decoder model to obtain the speed profile to be followed by an autonomous electric truck platoon considering various constraints such as the available state of charge (SOC) in the batteries along with other vehicles and road conditions while ensuring that the platoon is string stable. To ensure that the framework is suitable for long-haul highway operation, the model has been trained using various known highway drive cycles. Encoder-decoder models were trained and hyperparameter tuning was performed for the same. Finally, the most suitable model has been chosen for the application. For testing the entire framework, drive cycle/speed prediction corresponding to different desired SOC profiles has been presented. A case study showing the relevance of the proposed framework in predicting the drive cycle on various routes and its impact on taking critical policy decisions during the planning of electric truck platoons has also been presented. This study would help to efficiently plan the feasible routes for electric trucks considering multiple constraints such as battery capacity, expected discharge rate, charging infrastructure availability, route length/travel time, and other on-road operating conditions while also maintaining stability.

Skin Cancer Segmentation using CNN

This groundbreaking research introduces a comprehensive strategy for advancing medical image segmentation, merging two pivotal concepts to significantly enhance both the accuracy and efficiency of the segmentation process. The first component of our approach involves the integration of polar transformations as a preprocessing step applied to the original dataset. This transformative technique is designed to address the challenges associated with segmenting single structures of elliptical shape in medical images, such as organs (e.g., heart and kidneys), skin lesions, polyps, and various abnormalities. By centering the polar transformation on the object's focal point, a reduction in dimensionality is achieved, coupled with a distinct separation of segmentation and localization tasks. Two distinct methodologies for selecting an optimal polar origin are proposed: one involving estimation through a segmentation neural network trained on non-polar images, and the other employing a dedicated neural network trained to pre dict the optimal origin. The second key element of our approach is around the integration of the DoubleU-Net architecture, a powerful encoder-decoder model specifically designed for the task of semantic image segmentation. DoubleU- Net is a group of two U-Net architectures, each with a specific purpose. The initial U-Net is pre-trained on VGG-19 as the encoder and uses features learned from ImageNet to provide efficient information transfer. In order to store more semantic information and content, a second U-Net was added to the base to enhance the capabilities of the network. Join Atrous Spatial Pyramid Pooling (ASPP) to develop network data extraction content. The combination of DoubleU-Net architecture and joint transformation as a step forward shows good segmentation performance in different clinical tasks, including liver segmentation, polyp detection vision, skin segmentation, and epicardial fat tissue segmentation. It shows that various medical projects, including various diagnostic methods such as colonoscopy, dermoscopy, microscopy, have a positive impact on the plan. More importantly, the method performs well in difficult cases, such as the segmentation of small and flat polyps in CVC-ClinicDB and the 2015 subset of the MICCAI Automated Polyp Detection dataset. The results demonstrate the accuracy and generality of the combination, making it the best way to evaluate medical images in context; Our study has revealed a new method of skin cancer diagnosis that combines the power of deep learning with innovation. Advanced technology. The combination of dual U-Net architecture and polar coordinate transformation not only improves the accuracy of classification of lesions but also improves the robustness of the model to changes in image features. This study contributes to the development of computer-aided diagnostic systems for early diagnosis. Experimental results show that our method provides good accuracy, sensitivity, and specificity in detecting malignant and benign tumors. Additionally, we are conducting ablation studies to determine the contribution of each presentation and treatment of skin cancer to ultimately benefit patients and determine these benefits. We also apply the transformation of the joint as the first step to improve the discrimination ability of the model. This mechanical change effectively reduces the impact caused by changes in wound size, shape, and direction by displaying the original image of the polar system. By standardizing the representation of skin diseases, polar transformation improves the model's ability to generalize across different data sets and improves overall performance.

OcularSeg: Accurate and Efficient Multi-Modal Ocular Segmentation in Non-Constrained Scenarios

Multi-modal ocular biometrics has recently garnered significant attention due to its potential in enhancing the security and reliability of biometric identification systems in non-constrained scenarios. However, accurately and efficiently segmenting multi-modal ocular traits (periocular, sclera, iris, and pupil) remains challenging due to noise interference or environmental changes, such as specular reflection, gaze deviation, blur, occlusions from eyelid/eyelash/glasses, and illumination/spectrum/sensor variations. To address these challenges, we propose OcularSeg, a densely connected encoder–decoder model incorporating eye shape prior. The model utilizes Efficientnetv2 as a lightweight backbone in the encoder for extracting multi-level visual features while minimizing network parameters. Moreover, we introduce the Expectation–Maximization attention (EMA) unit to progressively refine the model’s attention and roughly aggregate features from each ocular modality. In the decoder, we design a bottom-up dense subtraction module (DSM) to amplify information disparity between encoder layers, facilitating the acquisition of high-level semantic detailed features at varying scales, thereby enhancing the precision of detailed ocular region prediction. Additionally, boundary- and semantic-guided eye shape priors are integrated as auxiliary supervision during training to optimize the position, shape, and internal topological structure of segmentation results. Due to the scarcity of datasets with multi-modal ocular segmentation annotations, we manually annotated three challenging eye datasets captured in near-infrared and visible light scenarios. Experimental results on newly annotated and existing datasets demonstrate that our model achieves state-of-the-art performance in intra- and cross-dataset scenarios while maintaining efficient execution.

Transformer Models for Brazilian Portuguese Question Generation: An Experimental Study

Unlike tasks such as translation or summarization, generating meaningful questions necessitates a profound understanding of context, semantics, and syntax. This complexity arises from the need to not only comprehend the given text comprehensively but also infer information gaps, identify relevant entities, and construct syntactically and semantically correct interrogative sentences. We address this challenge by proposing an experimental fine-tuning approach for encoder-decoder models (T5, FLAN-T5, and BART-PT) tailored explicitly for Brazilian Portuguese question generation. Our study involves fine-tuning these models on the SQUAD-v1.1 dataset and subsequent evaluation, also on SQUAD-v1.1. Through our experimental endeavors, BART returned a higher result in all the ROUGE metrics, as ROUGE-1 0.46, ROUGE-2 0.24, and ROUGE-L 0.43, suggesting a higher lexical similarity in the questions generated, and it is comparable to the results of the question generation task for the English language. We explored how these advancements can significantly enhance the precision and quality of the question generation task in Brazilian Portuguese, bridging the gap between training data and the intricacies of interrogative sentence construction.

MuLX-QA: Classifying Multi-Labels and Extracting Rationale Spans in Social Media Posts

While social media platforms play an important role in our daily lives in obtaining the latest news and trends from across the globe, they are known to be prone to widespread proliferation of harmful information in different forms leading to misconceptions among the masses. Accordingly, several prior works have attempted to tag social media posts with labels/classes reflecting their veracity, sentiments, hate content, and so on. However, in order to have a convincing impact, it is important to additionally extract the post snippets on which the labelling decision is based. We call such a post snippet the rationale . These rationales significantly improve human trust and debuggability of the predictions, especially when detecting misinformation or stigmas from social media posts. These rationale spans or snippets are also helpful in post-classification social analysis, such as for finding out the target communities in hate-speech, or for understanding the arguments or concerns against the intake of vaccines. Also it is observed that a post may express multiple notions of misinformation, hate, sentiment, and the like. Thus, the task of determining (one or multiple) labels for a given piece of text, along with the text snippets explaining the rationale behind each of the identified labels is a challenging multi-label, multi-rationale classification task, which is still nascent in the literature. While transformer -based encoder-decoder generative models such as BART and T5 are well suited for the task, in this work we show how a relatively simpler encoder-only discriminative question-answering (QA) model can be effectively trained using simple template-based questions to accomplish the task. We thus propose MuLX-QA and demonstrate its utility in producing (label, rationale span) pairs in two different settings: multi-class (on the HateXplain dataset related to hate speech on social media), and multi-label (on the CAVES dataset related to COVID-19 anti-vaccine concerns). MuLX-QA outperforms heavier generative models in both settings. We also demonstrate the relative advantage of our proposed model MuLX-QA over strong baselines when trained with limited data. We perform several ablation studies, and experiments to better understand the effect of training MuLX-QA with different question prompts, and draw interesting inferences. Additionally, we show that MuLX-QA is effective on social media posts in resource-poor non-English languages as well. Finally, we perform a qualitative analysis of our model predictions and compare them with those of our strongest baseline.

A Test Statistic Estimation-Based Approach for Establishing Self-Interpretable CNN-Based Binary Classifiers.

Interpretability is highly desired for deep neural network-based classifiers, especially when addressing high-stake decisions in medical imaging. Commonly used post-hoc interpretability methods have the limitation that they can produce plausible but different interpretations of a given model, leading to ambiguity about which one to choose. To address this problem, a novel decision-theory-inspired approach is investigated to establish a self-interpretable model, given a pre-trained deep binary black-box medical image classifier. This approach involves utilizing a self-interpretable encoder-decoder model in conjunction with a single-layer fully connected network with unity weights. The model is trained to estimate the test statistic of the given trained black-box deep binary classifier to maintain a similar accuracy. The decoder output image, referred to as an equivalency map, is an image that represents a transformed version of the to-be-classified image that, when processed by the fixed fully connected layer, produces the same test statistic value as the original classifier. The equivalency map provides a visualization of the transformed image features that directly contribute to the test statistic value and, moreover, permits quantification of their relative contributions. Unlike the traditional post-hoc interpretability methods, the proposed method is self-interpretable, quantitative. Detailed quantitative and qualitative analyses have been performed with three different medical image binary classification tasks.

Temperature forecasting for single-phase immersion cooling system based on machine learning

High-heat-flux-density data centres face significant thermal management challenges, particularly those for 5G and AI applications. Elevated processor temperatures lead to frequent thermal throttling and non-uniform thermal stresses, negatively impacting server performance and reliability. This highlights the need for effective thermal management strategies. Machine learning-based temperature prediction algorithms have shown promise in proactively managing the thermal condition of the system. In his study, a Raspberry Pi 4 Model B+ was immersed in mineral oil to simulate the workloads based on cryptography, Fast Fourier Transform (FFT) and data analytics. The discrete wavelet transform technique is used to mitigate the impact of noise interference in the experimental data. The results show that the attention-based LSTM encoder-decoder model (LSTM-ED-Attention) coupled with the entropy minimisation strategy outperformed the baseline LSTM encoder-decoder model by 22, 15 and 20% for prediction horizons of 10, 30 and 60 seconds, respectively.

Linguistic based emotion analysis using softmax over time attention mechanism.

Recognizing the real emotion of humans is considered the most essential task for any customer feedback or medical applications. There are many methods available to recognize the type of emotion from speech signal by extracting frequency, pitch, and other dominant features. These features are used to train various models to auto-detect various human emotions. We cannot completely rely on the features of speech signals to detect the emotion, for instance, a customer is angry but still, he is speaking at a low voice (frequency components) which will eventually lead to wrong predictions. Even a video-based emotion detection system can be fooled by false facial expressions for various emotions. To rectify this issue, we need to make a parallel model that will train on textual data and make predictions based on the words present in the text. The model will then classify the type of emotions using more comprehensive information, thus making it a more robust model. To address this issue, we have tested four text-based classification models to classify the emotions of a customer. We examined the text-based models and compared their results which showed that the modified Encoder decoder model with attention mechanism trained on textual data achieved an accuracy of 93.5%. This research highlights the pressing need for more robust emotion recognition systems and underscores the potential of transfer models with attention mechanisms to significantly improve feedback management processes and the medical applications.

Yes, I am afraid of the sharks and also wild lions!: A multitask framework for enhancing dialogue generation via knowledge and emotion grounding

Current end-to-end neural conversation models inherently lack the capability to generate coherently engaging responses. Efforts to boost informativeness have an adversarial effect on emotional and factual accuracy, as validated by several sequence-based models. While these issues can be alleviated by access to emotion labels and background knowledge, there is no guarantee of relevance and informativeness in the generated responses. In real dialogue corpus, informative words like named entities, and words that carry specific emotions can often be infrequent and hard to model, and one primary challenge of the dialogue system is how to promote the model’s capability of generating high-quality responses with those informative words. Furthermore, earlier approaches depended on straightforward concatenation techniques that lacked robust representation capabilities in order to account for human emotions. To address this problem, we propose a novel multitask hierarchical encoder–decoder model, which can enhance the multi-turn dialogue response generation by incorporating external textual knowledge and relevant emotions. Experimental results on a benchmark dataset indicate that our model is superior over competitive baselines concerning both automatic and human evaluation.

A Comparative Study of Transformer-based Models for Text Summarization of News Articles

Transformer-based models such as GPT, T5, BART, and PEGASUS have made substantial progress in text summarization, a sub-domain of natural language processing that entails extracting important information from lengthy texts. The main objective of this research was to conduct a comparative analysis of these four transformer-based models based on their performance in text summarization of news articles. In achieving this objective, the transformer models pre-trained on extensive datasets were fine-tuned on the CNN/DailyMail dataset using a low learning rate to preserve the learned representations. The T5 transformer records the highest scores of 35.12, 22.75, 32.82, and 28.59 in ROUGE-1, ROUGE-2, ROUGE-L, and ROUGE-Lsum respectively, surpassing GPT, BART, and PEGASUS across all ROUGE metrics. The findings deduced from this study establish the proficiency of encoder-decoder models such as T5 in summary generation. Furthermore, the findings also demonstrated that the fine-tuning process's effectiveness in pre-trained models is improved when the pre-training objective closely aligns with the downstream task.

Prediction of Load Demand in Home Area Network Using LSTM

Load forecasting is one of the most important tools for the energy management system in the modern world. Forecasting power consumption and load demand calculation became an interesting topic for the stakeholders in the electricity market. Decision-making in purchasing and generating electric power, load switching, and demand side management are dependent on load forecasting. This research work focuses on the prediction of power consumption in a Home Area Network (HAN) using time series forecasting methods like Long short-term memory (LSTM) neural Networks. Our goal was to design a model that could precisely forecast the electrical load required in a Home Area Network. Utility companies and homeowners can utilize this model to better plan and control their power usage. It can assist in lowering the likelihood of power outages and enhancing the effectiveness of the electrical system. The Root Mean Squared Error (RMSE) is used as the performance measure in our work. The dataset considered is from the UCI repository with 350400 rows and 4 parameters such as active power, and 3 submetering values, to achieve realistic data for training the model used the encoder-decoder LSTM model for load prediction and achieved an RMSE value of 23 and an MAE value of 18.6

An encoder-decoder ConvLSTM surrogate model for simulating geological CO2 sequestration with dynamic well controls

In Geological Carbon Sequestration (GCS), effectively managing the project requires predicting state variables such as pressure and saturation. However, numerical simulation of multiphase flow in subsurface porous media involves solving large linear algebra systems, resulting in a substantial computational burden. This can make it impractical for real-time history matching or optimization. Meanwhile, deep learning-based surrogate models are emerging as fast and accurate approximators. This study delves into the application of the Encoder-Decoder Convolutional Long Short-Term Memory (ED-ConvLSTM) neural network for predicting the complex evolution of state variables under dynamic CO2 injection schemes. Inception blocks enhanced with light-weighted attention modules are introduced in the encoder to extract high-dimensional input features. ConvLSTM is employed to propagate spatial temporal information in the low-dimensional latent space. Further, progressive upsampling blocks are used to reconstruct the latent features for the desired output. Instead of taking discrete time steps as an input feature, the proposed network captures the dynamic dependencies with the inherent ConvLSTM cell. The network has access to data at only portion of the initial time steps during training stage, while it is used to predict the state variables at unseen time steps during testing stage. Results show that the network can produce excellent predictions for both pressure and saturation, even at unseen future time steps. The remarkable generalizability to different geological permeability fields is also evaluated. ED-ConvLSTM outperforms the standard U-Net by far, especially when predicting beyond the training time period. These numerical experiments demonstrate the advantages of ED-ConvLSTM in terms of prediction accuracy, extrapolability and generalizability. This study highlights the importance of incorporating recurrent connections into the deep neural networks for simulating time-dependent multiphase flow problems. The proposed methodology has great potential in GCS surrogate modeling and offers a possible approach for real-time optimization of CO2 injection.

Abstract LB241: FateAI: Analysis of circulating DNA fragmentomics with an artificial intelligence model for cancer screening

Abstract Cell-free DNA (cf-DNA) circulates in the blood due to cell death, which can result from normal cellular processes or specific diseases such as cancer and others. The set of epigenetic footprints of cf-DNA, such as nucleosome organization, end-motifs distribution, fragmentation pattern, and DNA methylation signatures, constitute a key factor in non-tumor-informed liquid biopsies because the epigenetic status is highly tissue-specific and can distinguish cancer types and cancer subtypes. We developed Fragment Analysis for Tumor Evaluation with Artificial Intelligence (Fate-AI). It integrates multiple epigenetic footprints extracted from Low Pass Whole Genome Sequencing to train an encoder-decoder machine learning model performing a multidimensional embedding of the feature along the genome for tumor detection and tissue of origin classification. Peripheral blood plasma samples, 10 mL, from 167 treatment-naive patients with Prostate Cancer (n=35), Lung Cancer (n=33), and Colon Cancer (n=99) were collected in multiple Italian cancer centers. Blood samples from 62 healthy donors were used as controls. In a leave-one-out cross-validation evaluation, Fate-AI outperforms state-of-the-art liquid biopsy approaches based on fragmentomics such as DELFI (Cristiano et al., Nature 2019) and GRIFFIN (Doeble et al. Nat. Comm. 2020) (Table 1). The model trained with the fragmentomics features of our cohort was applied to an external colon cancer encompassing 43 tumor samples and 61 healthy cases (Hallermayr et al. J. Hematol. Oncol. 2022), Fate-AI obtained an AUC of 0.923 vs. an AUC of 0.722 for DELFI and 0.695 for GRIFFIN. In conclusion, Fate-AI is a novel, highly accurate AI-based tool that outperforms current genomewide non-tumor-informed cancer detection approaches. Table 1. AUC, sensitivity and specificity of Fate-AI in cancer detection, and its comparison with state-of-the-art methods. Cancer AUC Sensitivity Specificity Fate-AI DELFI GRIFFIN Fate-AI DELFI GRIFFIN Fate-AI DELFI GRIFFIN Colon 0.973 0.852 0.82 0.961 0.706 0.902 0.891 0.873 0.582 Lung 0.905 0.905 0.884 0.939 0.788 0.97 0.982 0.855 0.661 Prostate 0.992 0.787 0.94 0.943 0.943 0.829 0.98 0.639 0.608 Citation Format: Antonio De Falco, Piera Grisolia, Rossella Tufano, Cinzia Graziano, Marianna Scrima, Clara Iannarone, Marco Bocchetti, Michele Falco, Francesca Carlino, Teresa Noviello, Anna Ceccarelli, Gabriella Misso, Chiara Tammaro, Maria Carminia Della Cort, Floriana Morgillo, Fortunato Ciardiello, Maria Rosaria Rizzo, Alfonso Fiorelli, Noemi Maria Giorgiano, Pasquale Vitale, Raffaele Addeo, Michele Orditura, Stefano Forte, Raffaella Giuffrida, Michele Caraglia, Michele Ceccarelli. FateAI: Analysis of circulating DNA fragmentomics with an artificial intelligence model for cancer screening [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 2 (Late-Breaking, Clinical Trial, and Invited Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(7_Suppl):Abstract nr LB241.

Big Data Analysis and User Behavior Prediction of Social Networks Based on Artificial Neural Network

The prediction of user behavior in social networks is of great significance for understanding user dynamics, personalized recommendation, and information dissemination. With the development of artificial intelligence (AI) technology, especially the application of artificial neural networks in big data analysis, new solutions and technical means have been provided for the analysis of user behavior in social networks. This study constructs a social network user behavior prediction model based on artificial neural networks. The article first reviews related research, establishes a research framework, and then describes in detail the functional structure of the user behavior prediction system. The key data structure design is meticulously constructed and the model is built through steps such as data preprocessing module, 3D feature frame construction module, feature mapping module, and feature prediction module. In addition, the article also constructs a group behavior theme probability prediction model based on an improved encoder-decoder model, which further enhances the understanding of group behavior of users in social networks. Through comparative experiments, the model proposed in this paper demonstrates its effectiveness in predicting individual and group user behaviors. The experimental results show that the model can accurately predict the behavior patterns of users in social networks and is superior to existing methods in terms of prediction accuracy and computational efficiency. This research shows that artificial neural networks are a powerful tool for analyzing big data in social networks and for predicting user behavior. The success of the study verifies the effectiveness of the model, provides a new technical path for future analysis of user behavior in social networks, and is expected to be widely applied in areas such as personalized recommendation and information dissemination analysis.

Attention-Based Multi-Layered Encoder-Decoder Model for Summarizing Non-Interactive User-Based Videos

Video summarization extracts the relevant contents from a video and presents the entire content of the video in a compact and summarized form. User based video summarization, can summarize a video as per the requirement of the user. In this work, a non interactive and a perception-based video summarization technique is proposed that makes use of attention mechanism to capture user’s interest and extract relevant keyshots in temporal sequence from the video content. Here, video summarization has been articulated as a sequence-to-sequence learning problem and a supervised method has been proposed for summarization of the video. Adding layers to the existing network makes it deeper, enables higher level of abstraction and facilitates better feature extraction. Therefore, the proposed model uses a multi-layered, deep summarization encoder-decoder network (MLAVS), with attention mechanism to select final keyshots from the video. The contextual information of the video frames is encoded using a multi-layered Bidirectional Long Short-Term Memory network (BiLSTM) as the encoder. To decode, a multi-layered attention-based Long Short-Term memory (LSTM) using a multiplicative score function is employed. The experiments are performed on the benchmark TVSum dataset and the results obtained are compared with recent works. The results show considerable improvement and clearly demonstrate the efficacy of this methodology against most of the other available state-of-art methods.

A Case Study on the Diminishing Popularity of Encoder-Only Architectures in Machine Learning Models

This paper examines the shift from encoder-only to decoder and encoder-decoder models in machine learning, highlighting the decline in popularity of encoder-only architectures. It explores the reasons behind this trend, such as the advancements in decoder models that offer superior generative capabilities, flexibility across various domains, and enhancements in unsupervised learning techniques. The study also discusses the role of prompting techniques in simplifying model architectures and enhancing model versatility. By analyzing the evolution, applications, and shifting preferences within the research community and industry, this paper aims to provide insights into the changing landscape of machine learning model architectures.

Existence Is Chaos: Enhancing 3D Human Motion Prediction with Uncertainty Consideration

Human motion prediction is consisting in forecasting future body poses from historically observed sequences. It is a longstanding challenge due to motion's complex dynamics and uncertainty. Existing methods focus on building up complicated neural networks to model the motion dynamics. The predicted results are required to be strictly similar to the training samples with L2 loss in current training pipeline. However, little attention has been paid to the uncertainty property which is crucial to the prediction task. We argue that the recorded motion in training data could be an observation of possible future, rather than a predetermined result. In addition, existing works calculate the predicted error on each future frame equally during training, while recent work indicated that different frames could play different roles. In this work, a novel computationally efficient encoder-decoder model with uncertainty consideration is proposed, which could learn proper characteristics for future frames by a dynamic function. Experimental results on benchmark datasets demonstrate that our uncertainty consideration approach has obvious advantages both in quantity and quality. Moreover, the proposed method could produce motion sequences with much better quality that avoids the intractable shaking artefacts. We believe our work could provide a novel perspective to consider the uncertainty quality for the general motion prediction task and encourage the studies in this field. The code will be available in https://github.com/Motionpre/Adaptive-Salient-Loss-SAGGB.

Winnie: Task-Oriented Dialog System with Structure-Aware Contrastive Learning and Enhanced Policy Planning

Pre-trained encoder-decoder models are widely applied in Task-Oriented Dialog (TOD) systems on the session level, mainly focusing on modeling the dialog semantic information. Dialogs imply structural information indicating the interaction among user utterances, belief states, database search results, system acts and responses, which is also crucial for TOD systems. In addition, for the system acts, additional pre-training and datasets are considered to improve their accuracies, undoubtedly introducing a burden. Therefore, a novel end-to-end TOD system named Winnie is proposed in this paper to improve the TOD performance. First, to make full use of the intrinsic structural information, supervised contrastive learning is adopted to narrow the gap in the representation space between text representations of the same category and enlarge the overall continuous representation margin between text representations of different categories in dialog context. Then, a system act classification task is introduced for policy optimization during fine-tuning. Empirical results show that Winnie substantially improves the performance of the TOD system. By introducing the supervised contrastive and system act classification losses, Winnie achieves state-of-the-art results on benchmark datasets, including MultiWOZ2.2, In-Car, and Camrest676. Their end-to-end combined scores are improved by 3.2, 1.9, and 1.1 points, respectively.