Types Of Neural Architectures Research Articles

Is neuro-symbolic AI meeting its promises in natural language processing? A structured review

Advocates for Neuro-Symbolic Artificial Intelligence (NeSy) assert that combining deep learning with symbolic reasoning will lead to stronger AI than either paradigm on its own. As successful as deep learning has been, it is generally accepted that even our best deep learning systems are not very good at abstract reasoning. And since reasoning is inextricably linked to language, it makes intuitive sense that Natural Language Processing (NLP), would be a particularly well-suited candidate for NeSy. We conduct a structured review of studies implementing NeSy for NLP, with the aim of answering the question of whether NeSy is indeed meeting its promises: reasoning, out-of-distribution generalization, interpretability, learning and reasoning from small data, and transferability to new domains. We examine the impact of knowledge representation, such as rules and semantic networks, language structure and relational structure, and whether implicit or explicit reasoning contributes to higher promise scores. We find that systems where logic is compiled into the neural network lead to the most NeSy goals being satisfied, while other factors such as knowledge representation, or type of neural architecture do not exhibit a clear correlation with goals being met. We find many discrepancies in how reasoning is defined, specifically in relation to human level reasoning, which impact decisions about model architectures and drive conclusions which are not always consistent across studies. Hence we advocate for a more methodical approach to the application of theories of human reasoning as well as the development of appropriate benchmarks, which we hope can lead to a better understanding of progress in the field. We make our data and code available on github for further analysis.11 https://github.com/kyleiwaniec/neuro-symbolic-ai-systematic-review

Interpretable deep learning to map diagnostic texts to ICD-10 codes

BackgroundAutomatic extraction of morbid disease or conditions contained in Death Certificates is a critical process, useful for billing, epidemiological studies and comparison across countries. The fact that these clinical documents are written in regular natural language makes the automatic coding process difficult because, often, spontaneous terms diverge strongly from standard reference terminology such as the International Classification of Diseases (ICD). ObjectiveOur aim is to propose a general and multilingual approach to render Diagnostic Terms into the standard framework provided by the ICD. We have evaluated our proposal on a set of clinical texts written in French, Hungarian and Italian. MethodsICD-10 encoding is a multi-class classification problem with an extensive (thousands) number of classes. After considering several approaches, we tackle our objective as a sequence-to-sequence task. According to current trends, we opted to use neural networks. We tested different types of neural architectures on three datasets in which Diagnostic Terms (DTs) have their ICD-10 codes associated. Results and conclusionsOur results give a new state-of-the art on multilingual ICD-10 coding, outperforming several alternative approaches, and showing the feasibility of automatic ICD-10 prediction obtaining an F-measure of 0.838, 0.963 and 0.952 for French, Hungarian and Italian, respectively. Additionally, the results are interpretable, providing experts with supporting evidence when confronted with coding decisions, as the model is able to show the alignments between the original text and each output code.

Comparative Study of Various Neural Network Architectures for MPEG-4 Video Traffic Prediction

<p>Network traffic as it is VBR in nature exhibits strong correlations which make it suitable for prediction. Real-time forecasting of network traffic load accurately and in a computationally efficient manner is the key element of proactive network management and congestion control. This paper comments on the MPEG-4 video traffic predictions evaluated by different types of neural network architectures and compares the performance of the same in terms of mean square error for the same video frames. For that three types of neural architectures are used namely Feed forward, Cascaded Feed forward and Time Delay Neural Network. The results show that cascade feed forward network produces minimum error as compared to other networks. This paper also compares the results of traditional prediction method of averaging of frames for future frame prediction with neural based methods. The experimental results show that nonlinear prediction based on NNs is better suited for traffic prediction purposes than linear forecasting models.</p>

学習とロボット 力学系に基づく自律移動ロボットの行動学習

This paper describes an experimental results based on our prior-proposed scheme : learning of sensory-based, goal-directed navigation. The emphasis is that learning a task-based behavior could be formulated as an embedding problem of dynamical systems : desired trajectories in a task space should be embedded into an adequate sensory-based internal state space so that an unique mapping from the internal state space to the motor command could be established. In this study, two types of neural architecture were tested for constructing such an adequate internal state space based on sensory information. A simple homing task was conducted on Feed Forward Neural Network that is fed with a regression of sensory sequence as time-delayed manner. A task of complex cyclic routing was conducted on Recurrent Neural Network (RNN) . In both cases, sufficient supervised training of robots generated rigid internal structure of attractor dynamics which realized desired navigations in robust manner, even against limited environmental changes as well as miscellaneous noises in the real world. It was confirmed that RNN architecture has more flexiblity adapting to various tasks.