Task Dataset Research Articles

CLIN-X: pre-trained language models and a study on cross-task transfer for concept extraction in the clinical domain.

The field of natural language processing (NLP) has recently seen a large change toward using pre-trained language models for solving almost any task. Despite showing great improvements in benchmark datasets for various tasks, these models often perform sub-optimal in non-standard domains like the clinical domain where a large gap between pre-training documents and target documents is observed. In this article, we aim at closing this gap with domain-specific training of the language model and we investigate its effect on a diverse set of downstream tasks and settings. We introduce the pre-trained CLIN-X (Clinical XLM-R) language models and show how CLIN-X outperforms other pre-trained transformer models by a large margin for 10 clinical concept extraction tasks from two languages. In addition, we demonstrate how the transformer model can be further improved with our proposed task- and language-agnostic model architecture based on ensembles over random splits and cross-sentence context. Our studies in low-resource and transfer settings reveal stable model performance despite a lack of annotated data with improvements of up to 47 F1 points when only 250 labeled sentences are available. Our results highlight the importance of specialized language models, such as CLIN-X, for concept extraction in non-standard domains, but also show that our task-agnostic model architecture is robust across the tested tasks and languages so that domain- or task-specific adaptations are not required. The CLIN-X language models and source code for fine-tuning and transferring the model are publicly available at https://github.com/boschresearch/clin_x/ and the huggingface model hub.

Higher-dimensional neural representations predict better episodic memory.

Episodic memory enables humans to encode and later vividly retrieve information about our rich experiences, yet the neural representations that support this mental capacity are poorly understood. Using a large fMRI dataset (n = 468) of face-name associative memory tasks and principal component analysis to examine neural representational dimensionality (RD), we found that the human brain maintained a high-dimensional representation of faces through hierarchical representation within and beyond the face-selective regions. Critically, greater RD was associated with better subsequent memory performance both within and across participants, and this association was specific to episodic memory but not general cognitive abilities. Furthermore, the frontoparietal activities could suppress the shared low-dimensional fluctuations and reduce the correlations of local neural responses, resulting in greater RD. RD was not associated with the degree of item-specific pattern similarity, and it made complementary contributions to episodic memory. These results provide a mechanistic understanding of the role of RD in supporting accurate episodic memory.

Microscopic nuclei classification, segmentation, and detection with improved deep convolutional neural networks (DCNN)

BackgroundNuclei classification, segmentation, and detection from pathological images are challenging tasks due to cellular heterogeneity in the Whole Slide Images (WSI).MethodsIn this work, we propose advanced DCNN models for nuclei classification, segmentation, and detection tasks. The Densely Connected Neural Network (DCNN) and Densely Connected Recurrent Convolutional Network (DCRN) models are applied for the nuclei classification tasks. The Recurrent Residual U-Net (R2U-Net) and the R2UNet-based regression model named the University of Dayton Net (UD-Net) are applied for nuclei segmentation and detection tasks respectively. The experiments are conducted on publicly available datasets, including Routine Colon Cancer (RCC) classification and detection and the Nuclei Segmentation Challenge 2018 datasets for segmentation tasks. The experimental results were evaluated with a five-fold cross-validation method, and the average testing results are compared against the existing approaches in terms of precision, recall, Dice Coefficient (DC), Mean Squared Error (MSE), F1-score, and overall testing accuracy by calculating pixels and cell-level analysis.ResultsThe results demonstrate around 2.6% and 1.7% higher performance in terms of F1-score for nuclei classification and detection tasks when compared to the recently published DCNN based method. Also, for nuclei segmentation, the R2U-Net shows around 91.90% average testing accuracy in terms of DC, which is around 1.54% higher than the U-Net model.ConclusionThe proposed methods demonstrate robustness with better quantitative and qualitative results in three different tasks for analyzing the WSI.

Considering Commonsense in Solving QA: Reading Comprehension with Semantic Search and Continual Learning

Unlike previous dialogue-based question-answering (QA) datasets, DREAM, multiple-choice Dialogue-based REAding comprehension exaMination dataset, requires a deep understanding of dialogue. Many problems require multi-sentence reasoning, whereas some require commonsense reasoning. However, most pre-trained language models (PTLMs) do not consider commonsense. In addition, because the maximum number of tokens that a language model (LM) can deal with is limited, the entire dialogue history cannot be included. The resulting information loss has an adverse effect on performance. To address these problems, we propose a Dialogue-based QA model with Common-sense Reasoning (DQACR), a language model that exploits Semantic Search and continual learning. We used Semantic Search to complement information loss from truncated dialogue. In addition, we used Semantic Search and continual learning to improve the PTLM’s commonsense reasoning. Our model achieves an improvement of approximately 1.5% over the baseline method and can thus facilitate QA-related tasks. It contributes toward not only dialogue-based QA tasks but also another form of QA datasets for future tasks.

Fake news spreaders profiling using N-grams of various types and SHAP-based feature selection

Complex learning approaches along with complicated and expensive features are not always the best or the only solution for Natural Language Processing (NLP) tasks. Despite huge progress and advancements in learning approaches such as Deep Learning (DL) and Transfer Learning (TL), there are many NLP tasks such as Text Classification (TC), for which basic Machine Learning (ML) classifiers perform superior to DL or TL approaches. Added to this, an efficient feature engineering step can significantly improve the performance of ML based systems. To check the efficacy of ML based systems and feature engineering on TC, this paper explores char, character sequences, syllables, word n-grams as well as syntactic n-grams as features and SHapley Additive exPlanations (SHAP) values to select the important features from the collection of extracted features. Voting Classifiers (VC) with soft and hard voting of four ML classifiers, namely: Support Vector Machine (SVM) with Linear and Radial Basis Function (RBF) kernel, Logistic Regression (LR), and Random Forest (RF) was trained and evaluated on Fake News Spreaders Profiling (FNSP) shared task dataset in PAN 2020. This shared task consists of profiling fake news spreaders in English and Spanish languages. The proposed models exhibited an average accuracy of 0.785 for both languages in this shared task and outperformed the best models submitted to this task.

A Study on Chinese-English Machine Translation Based on Transfer Learning and Neural Networks

The existing Chinese-English machine translation has problems such as inaccurate word translation and difficult translation of long sentences. To this end, this paper proposes a new machine translation model based on bidirectional Chinese-English translation incorporating translation knowledge and transfer learning, and the components of this model include a recurrent neural network-based translation quality assessment model and a self-focused network-based model. The experimental results demonstrate that our method works better on the dataset of machine translation quality assessment task for Chinese-English translation with more information, and the Pearson correlation coefficient of its quality assessment feature vector (such as word prediction vector representation) is higher.

Appearance-Based Loop Closure Detection via Locality-Driven Accurate Motion Field Learning

Loop closure detection (LCD) is of significant importance in simultaneous localization and mapping. It represents the robot&#x2019;s ability to recognize whether the current surrounding corresponds to a previously observed one. In this paper, we conduct this task in a two-step strategy: candidate frame selection and loop closure verification. The first step aims to search semantically similar images for the query one using features obtained by Key.Net with HardNet. Instead of adopting the traditional Bag-of-Words strategy, we utilize the aggregated selective match kernel to calculate the similarity between images. Subsequently, based on the potential property of motion field in the LCD scene, we propose a novel feature matching method, <i>i.e.</i>, exploiting the smoothness prior and learning the motion field for an image pair in a reproducing kernel Hilbert space (RKHS), to implement loop closure verification. Concretely, we formulate the learning problem into a Bayesian framework with latent variables indicating the true/false correspondences and a mixture model accounting for the distribution of data. Furthermore, we propose a locality-driven mechanism to enhance the local relevance of motion vectors and term the algorithm as locality-driven accurate motion field learning (LAL). To satisfy the requirement of efficiency in the LCD task, we use a sparse approximation and search a suboptimal solution for the motion field in the RKHS, termed as LAL&#x002A;. Extensive experiments are conducted on public datasets for feature matching and LCD tasks. The quantitative results demonstrate the effectiveness of our method over the current state-of-the-art, meanwhile showing its potential for long-term visual localization. The codes of LAL and LAL&#x002A; are publicly available at <uri>https://github.com/KN-Zhang/LAL</uri>.

COVID-DAI: A novel framework for COVID-19 detection and infection growth estimation using computed tomography images.

The COVID‐19 pandemic is spreading at a fast pace around the world and has a high mortality rate. Since there is no proper treatment of COVID‐19 and its multiple variants, for example, Alpha, Beta, Gamma, and Delta, being more infectious in nature are affecting millions of people, further complicates the detection process, so, victims are at the risk of death. However, timely and accurate diagnosis of this deadly virus can not only save the patients from life loss but can also prevent them from the complex treatment procedures. Accurate segmentation and classification of COVID‐19 is a tedious job due to the extensive variations in its shape and similarity with other diseases like Pneumonia. Furthermore, the existing techniques have hardly focused on the infection growth estimation over time which can assist the doctors to better analyze the condition of COVID‐19‐affected patients. In this work, we tried to overcome the shortcomings of existing studies by proposing a model capable of segmenting, classifying the COVID‐19 from computed tomography images, and predicting its behavior over a certain period. The framework comprises four main steps: (i) data preparation, (ii) segmentation, (iii) infection growth estimation, and (iv) classification. After performing the pre‐processing step, we introduced the DenseNet‐77 based UNET approach. Initially, the DenseNet‐77 is used at the Encoder module of the UNET model to calculate the deep keypoints which are later segmented to show the coronavirus region. Then, the infection growth estimation of COVID‐19 per patient is estimated using the blob analysis. Finally, we employed the DenseNet‐77 framework as an end‐to‐end network to classify the input images into three classes namely healthy, COVID‐19‐affected, and pneumonia images. We evaluated the proposed model over the COVID‐19‐20 and COVIDx CT‐2A datasets for segmentation and classification tasks, respectively. Furthermore, unlike existing techniques, we performed a cross‐dataset evaluation to show the generalization ability of our method. The quantitative and qualitative evaluation confirms that our method is robust to both COVID‐19 segmentation and classification and can accurately predict the infection growth in a certain time frame.Research highlights We present an improved UNET framework with a DenseNet‐77‐based encoder for deep keypoints extraction to enhance the identification and segmentation performance of the coronavirus while reducing the computational complexity as well.We propose a computationally robust approach for COVID‐19 infection segmentation due to fewer model parameters.Robust segmentation of COVID‐19 due to accurate feature computation power of DenseNet‐77.A module is introduced to predict the infection growth of COVID‐19 for a patient to analyze its severity over time.We present such a framework that can effectively classify the samples into several classes, that is, COVID‐19, Pneumonia, and healthy samples.Rigorous experimentation was performed including the cross‐dataset evaluation to prove the efficacy of the presented technique

TAGLETS: A System for Automatic Semi-Supervised Learning with Auxiliary Data

Machine learning practitioners often have access to a spectrum of data: labeled data for the target task (which is often limited), unlabeled data, and auxiliary data, the many available labeled datasets for other tasks. We describe TAGLETS, a system built to study techniques for automatically exploiting all three types of data and creating high-quality, servable classifiers. The key components of TAGLETS are: (1) auxiliary data organized according to a knowledge graph, (2) modules encapsulating different methods for exploiting auxiliary and unlabeled data, and (3) a distillation stage in which the ensembled modules are combined into a servable model. We compare TAGLETS with state-of-the-art transfer learning and semi-supervised learning methods on four image classification tasks. Our study covers a range of settings, varying the amount of labeled data and the semantic relatedness of the auxiliary data to the target task. We find that the intelligent incorporation of auxiliary and unlabeled data into multiple learning techniques enables TAGLETS to match-and most often significantly surpass-these alternatives. TAGLETS is available as an open-source system at github.com/BatsResearch/taglets.

A Survey on Deep Learning for Software Engineering

In 2006, Geoffrey Hinton proposed the concept of training “Deep Neural Networks (DNNs)” and an improved model training method to break the bottleneck of neural network development. More recently, the introduction of AlphaGo in 2016 demonstrated the powerful learning ability of deep learning and its enormous potential. Deep learning has been increasingly used to develop state-of-the-art software engineering (SE) research tools due to its ability to boost performance for various SE tasks. There are many factors, e.g., deep learning model selection, internal structure differences, and model optimization techniques, that may have an impact on the performance of DNNs applied in SE. Few works to date focus on summarizing, classifying, and analyzing the application of deep learning techniques in SE. To fill this gap, we performed a survey to analyze the relevant studies published since 2006. We first provide an example to illustrate how deep learning techniques are used in SE. We then conduct a background analysis (BA) of primary studies and present four research questions to describe the trend of DNNs used in SE (BA), summarize and classify different deep learning techniques (RQ1), and analyze the data processing including data collection, data classification, data pre-processing, and data representation (RQ2). In RQ3, we depicted a range of key research topics using DNNs and investigated the relationships between DL-based model adoption and multiple factors (i.e., DL architectures, task types, problem types, and data types). We also summarized commonly used datasets for different SE tasks. In RQ4, we summarized the widely used optimization algorithms and provided important evaluation metrics for different problem types, including regression, classification, recommendation, and generation. Based on our findings, we present a set of current challenges remaining to be investigated and outline a proposed research road map highlighting key opportunities for future work.

Nirmaan: Dataset generation for multiclass scatterplot studies

This paper presents Nirmaan, an open-source web-based tool for generating synthetic datasets of multiclass blobs for use in research related to scatterplots. We demonstrate how to use Nirmaan to generate datasets in the context of a user study where users must determine the centers of each class, but this tool can be used to generate datasets for other scatterplot tasks as well.

Continual Learning Using Data Regeneration for Remote Sensing Scene Classification

When building a model with learning capability, the usual hypothesis is that the data of all the possible situations or tasks are available. Nevertheless, dealing with a massive number of tasks in a sequential manner necessitates preserving the previous tasks data and then retrain the model on it, which is infeasible. Another solution is to retrain the model only on the new task data, but, in this case, the model dramatically collapses when tested on the old tasks data due to the phenomenon known as catastrophic forgetting. To overcome this shortcoming, we propose a novel continual learning technique based on the learned tasks’ data auto-generation subnetworks. We sequentially train the proposed model on a set of classification tasks, where each task includes a certain number of remote sensing scenes or classes. The proposed neural network architecture encapsulates two trainable subnetworks. The first module adapts its weights by minimizing the discrimination error between the land-cover classes during the new task learning. In parallel, the second module learns how to reproduce data of the previous tasks by discovering the latent data structure of the new task dataset. Experiments are conducted on two scene datasets (Merced and Optimal31). The experimental results confirm the outperformance and robustness of the proposed model.

Contact Force Detection of Grinding Process Using Frequency Information and Differential Feature on Force Signal

In grinding tasks, the contact force has a significant impact on product surface quality. Therefore, force-sensing technology to detect contact force is important. Although force sensors are widely used for contact force detection, the response of the force sensor includes sensor-specific errors such as offset. In this paper, we propose a contact force detection method based on the combination of frequency information and the differential feature (Δ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</i> ) of the force signal. The use of high-frequency information reduces the influence of force sensor-specific errors. However, contact force detection using only high-frequency information causes a time delay in the detected value relative to the measured value depending to the frame size of time window used for frequency analysis. To reduce the time delay, high-frequency information and Δ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</i> are integrated by inputting them into an long short-term memory (LSTM)-based force detection model. To verify the effectiveness of the proposed method, we compared it with a force detection model based on an FNN and CNN on a dataset of plane grinding tasks. Consequently, the detection accuracy of the LSTM-based model was superior to that of the FNN and CNN models. Compared to the LSTM model using only high-frequency information as input, the detection accuracy was 26% higher when the error was small and 57% higher when the error was large. In addition, the time delay was reduced from 166 ms to 30 ms using Δ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</i> as the input. The frequency information and Δ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</i> are features calculated from the same force information dataset; therefore, no additional dataset is required.

Memory-Modulated Transformer Network for Heterogeneous Face Recognition

Heterogeneous face recognition (HFR) aims at matching face images across different domains. It is challenging due to the severe domain discrepancies and overfitting caused by small training datasets. Some researchers apply a &#x201C;recognition via generation&#x201D; strategy and propose to solve the problem by translating images from a given domain into the visual domain. However, in many HFR tasks such as near-infrablack HFR, there is no paiblack data, which makes it an unsupervised generation. Pose variations, background differences, and many other factors present challenges. Moreover, the generated results lack diversity since many previous works regard this image translation as a &#x201C;one-to-one&#x201D; generation task. Considering the information deficiency in the input images, we propose to formulate this image translation process as a &#x201C;one-to-many&#x201D; generation problem. Specifically, we introduce reference images to guide the generation process. We propose a memory module to explore the prototypical style patterns of the reference domain. After self-supervised updating, the memory items are attentively aggregated to represent the style information. Moreover, to subtly fuse the contents of input images with the style of reference images, we propose a novel style transformer module. Specifically, we crop the encoded input and reference feature maps into patches, and use the style transformer to establish long-range dependencies between the input and reference patches. Thus, the style of every input patch is transferblack based on those of the most relevant reference patches. Extensive experiments on multiple datasets for various HFR tasks, including NIR-VIS, thermal-VIS, sketch-photo, and gray-RGB, are conducted. The robustness and effectiveness of the proposed MMTN are demonstrated both quantitatively and qualitatively.

An Auto-Weighting Incremental Random Vector Functional Link Network for EEG-Based Driving Fatigue Detection

Recently, Electroencephalogram (EEG) has been receiving increasing attention in driving fatigue attention because it is generated by the neural activities of central nervous system and has been regarded as the gold standard to measure fatigue. However, most existing studies for EEG-based driving fatigue detection have some common limitations such as 1) using the batch learning mode and no incremental updating ability, 2) converting continuous fatigue indices into discrete levels which deviates far from the essence of fatigue detection, and 3) neglecting considering the different contributions of EEG feature dimensions in fatigue expression. To handle these problems, we propose an Auto-Weighting Incremental Random Vector Functional Link network (AWIRVFL) model for EEG-based driving fatigue detection, which simultaneously implements online regression prediction and incremental learning. Moreover, an auto-weighting variable is introduced to adaptively and quantitatively explore the importance of different feature dimensions. A novel optimization algorithm is proposed to solve the AWIRVFL objective function. Experiments were conducted on the SEED-VIG and sustained-attention driving task (SADT) data sets to validate the performance of AWIRVFL and the results demonstrated that AWIRVFL greatly outperforms the state-of-the-arts in terms of the two regression evaluation metrics, RMSE and MAPE. Moreover, the quantitative feature importance values are obtained.

Learning Global Average Attention Pooling (GAAP) on Resnet50 Backbone for Person Re-identification Problem

Person re-identification has been an extremely chal-lenging task in computer vision which has been seen as a success with deep learning approaches. Despite successful models, there are gaps in the form of unbalanced labels, poor resolution, uncertain bounding box annotations, occlusions, and unlabelled datasets. Previous methods applied deep learning approaches based on feature representation, metric learning, and ranking optimization. In this work, we propose Global Average Attention Pooling (GAAP) on Resnet50 applied on four benchmark Re-ID datasets for classification tasks. We also perform an extensive evaluation on the proposed Attention module with different deep learning pipelines as backbone architecture. The four benchmark person Re-ID datasets used is Market-1501, RAiD, Partial-iLIDS, and RPIfield. We computed cumulative matching characteristics (CMC) and mean Average Precision (mAP) as the performance evaluation parameters of the proposed against the state of the art. The results obtained have shown that the added attention layer has improved the overall recognition precision over the baselines.

Fast Classification and Action Recognition With Event-Based Imaging

The neuromorphic event cameras, which capture the optical changes of a scene, have drawn increasing attention due to their high speed and low power consumption. However, the event data are noisy, sparse, and nonuniform in the spatial-temporal domain with extremely high temporal resolution, making it challenging to process for traditional deep learning algorithms. To enable convolutional neural network models for event vision tasks, most methods encode events into point-cloud or voxel representations, but their performance still has much room for improvement. Additionally, as event cameras can only detect changes in the scene, relative movements can lead to misalignment, i.e., the same pixel may refer to different real-world points at different times. To this end, this work proposes the aligned compressed event tensor (ACE) as a novel event data representation, and a framework called branched event net (BET) for event-based vision under both static and dynamic scenes. We apply them on various datasets for object classification and action recognition tasks, and show that they surpass state-of-the-art methods by significant margins. Specifically, our method achieves 98.88% accuracy for the DVS128 action recognition task, and outperforms the second best method by large margins of 4.85%, 9.56% and 2.33% on N-Caltech101, DVSAction and NeuroIV datasets, respectively. Furthermore, the proposed ACE-BET is efficient, and achieves the fastest inference speed among various methods being tested.

Motor Imagery Classification Using Inter-Task Transfer Learning via a Channel-Wise Variational Autoencoder-Based Convolutional Neural Network.

Highly sophisticated control based on a brain-computer interface (BCI) requires decoding kinematic information from brain signals. The forearm is a region of the upper limb that is often used in everyday life, but intuitive movements within the same limb have rarely been investigated in previous BCI studies. In this study, we focused on various forearm movement decoding from electroencephalography (EEG) signals using a small number of samples. Ten healthy participants took part in an experiment and performed motor execution (ME) and motor imagery (MI) of the intuitive movement tasks (Dataset I). We propose a convolutional neural network using a channel-wise variational autoencoder (CVNet) based on inter-task transfer learning. We approached that training the reconstructed ME-EEG signals together will also achieve more sufficient classification performance with only a small amount of MI-EEG signals. The proposed CVNet was validated on our own Dataset I and a public dataset, BNCI Horizon 2020 (Dataset II). The classification accuracies of various movements are confirmed to be 0.83 (±0.04) and 0.69 (±0.04) for Dataset I and II, respectively. The results show that the proposed method exhibits performance increases of approximately 0.09~0.27 and 0.08~0.24 compared with the conventional models for Dataset I and II, respectively. The outcomes suggest that the training model for decoding imagined movements can be performed using data from ME and a small number of data samples from MI. Hence, it is presented the feasibility of BCI learning strategies that can sufficiently learn deep learning with a few amount of calibration dataset and time only, with stable performance.

Deep Transfer Learning for Wall Bulge Endpoints Regression for Autonomous Decoration Robots

Wall bulge maintenance and repairing is an essential task for autonomous decoration robots. The problem of the wall bulge endpoints regression refers to identifying the position of the wall bulge endpoints in spatial coordinates. This problem is of significant importance for autonomous decoration robots as these robots target automatic maintenance and repairing of wall bulges, they must automatically recognize where to start and stop the repairing process. Training deep convolutional neural networks for supervised computer vision tasks requires a large number of annotated images. Since gathering annotated images for this task is difficult, laborious, and time-consuming, we proposed a model for detecting the wall bulge endpoints position based on deep transfer learning. Our proposed model is capable of classifying the wall bulge into one of four classes according to its orientation. Deep transfer learning transfers the knowledge acquired by deep learning models trained for a specific task and domain to another different but related task and domain. Our proposed model is mainly based on deep convolutional neural networks pre-trained on large datasets for tasks of object classification and detection. We transfer the knowledge acquired by the model from these tasks to solve both problems in our new task.

Help Transformer Improve Performance in Automatic Mathematics Word Problem-Solving

Solving Mathematics Word Problem (MWP) is a basic ability of humanity, which can be mastered by most students at a young age. The existing artificial intelligence system is not good enough in numerical questions, like MWPs. The hard part of this problem is translating natural language sentences in MWP into mathematical expressions or equations. In recent researches, the Transformer network, which proved a great success in machine translation, is applied to automatic mathematic word problem-solving. While previous works have only shown the ability of Transformer model in MWP, how multiple factors such as encoding, decoding, and pre-training affect the performance of Transformer model has not received enough attention. The study is the first to examine the role of these factors experimentally. This paper proposes several methods to improve Transformer network performance in MWPs under the basis of previous studies, achieves higher accuracy compared to the previous state of the art. Pre-training on target tasks dataset improves the translation quality of the Transformer model greatly. Different token encoding and search algorithms also benefit prediction accuracy at the expense of more training and testing time.