Character-level Convolutional Networks Research Articles

Attention-Based Character-Word Hybrid Neural Networks With Semantic and Structural Information for Identifying of Urgent Posts in MOOC Discussion Forums

The MOOC Discussion Forum is the place where students and teachers communicate, often plagued by information overload and confusion. Posts that students used to express confusion and demanded teachers' attention are most likely to be overwhelmed by the amount of noise in the forum. Therefore, how to pay attention to urgent posts in time has become a critical problem to be solved. In this paper, we present a new hybrid neural network for identifying “urgent” posts that require immediate attention from instructors. We proposed a semantic and structure extraction part including convolutional neural network (CNN) and gated recurrent unit (GRU), which can simultaneously learn the semantic information and structural information of sentences. In addition, Due to a lot of noise such as spelling mistakes and emoticons in the forum comment text, we propose to use Character-level Convolutional Networks (Char-CNN) to capture these special information. Finally, the semantic and structural information learned by the semantic and structural extraction part is merged with the character information learned by Char-CNN, and the attention mechanism to learn their weights, the final representation of the sentence can be obtained. In our experiments, we achieve urgent posts classification with a micro F-score of 91.8%, 91.3% and 88.4% on the Stanford MOOCPosts dataset, outperforming the state-of-the-art approach by 1.8%, 2.4% and 1.5% respectively. This work can help teachers prioritize their responses and better manage numerous posts. Teachers can answer learner questions in a timely manner and help reduce dropout rates and improve completion rates.

Fully Character-Level Neural Machine Translation without Explicit Segmentation

Most existing machine translation systems operate at the level of words, relying on explicit segmentation to extract tokens. We introduce a neural machine translation (NMT) model that maps a source character sequence to a target character sequence without any segmentation. We employ a character-level convolutional network with max-pooling at the encoder to reduce the length of source representation, allowing the model to be trained at a speed comparable to subword-level models while capturing local regularities. Our character-to-character model outperforms a recently proposed baseline with a subword-level encoder on WMT’15 DE-EN and CS-EN, and gives comparable performance on FI-EN and RU-EN. We then demonstrate that it is possible to share a single character-level encoder across multiple languages by training a model on a many-to-one translation task. In this multilingual setting, the character-level encoder significantly outperforms the subword-level encoder on all the language pairs. We observe that on CS-EN, FI-EN and RU-EN, the quality of the multilingual character-level translation even surpasses the models specifically trained on that language pair alone, both in terms of the BLEU score and human judgment.