Abstract
Sentence processing takes place in real-time. Previous words in the sentence can influence the processing of the current word in the timescale of hundreds of milliseconds. Recent neurophysiological studies in humans suggest that the fronto-striatal system (frontal cortex, and striatum – the major input locus of the basal ganglia) plays a crucial role in this process. The current research provides a possible explanation of how certain aspects of this real-time processing can occur, based on the dynamics of recurrent cortical networks, and plasticity in the cortico-striatal system. We simulate prefrontal area BA47 as a recurrent network that receives on-line input about word categories during sentence processing, with plastic connections between cortex and striatum. We exploit the homology between the cortico-striatal system and reservoir computing, where recurrent frontal cortical networks are the reservoir, and plastic cortico-striatal synapses are the readout. The system is trained on sentence-meaning pairs, where meaning is coded as activation in the striatum corresponding to the roles that different nouns and verbs play in the sentences. The model learns an extended set of grammatical constructions, and demonstrates the ability to generalize to novel constructions. It demonstrates how early in the sentence, a parallel set of predictions are made concerning the meaning, which are then confirmed or updated as the processing of the input sentence proceeds. It demonstrates how on-line responses to words are influenced by previous words in the sentence, and by previous sentences in the discourse, providing new insight into the neurophysiology of the P600 ERP scalp response to grammatical complexity. This demonstrates that a recurrent neural network can decode grammatical structure from sentences in real-time in order to generate a predictive representation of the meaning of the sentences. This can provide insight into the underlying mechanisms of human cortico-striatal function in sentence processing.
Highlights
One of the most remarkable aspects of language processing is the rapidity with which it takes place
The stated goal of this research was to test the hypothesis that a recurrent network can encode the structure of sentences based on their closed class structure [33] and (a) use this information to perform thematic role assignment, (b) generate predictions about the real-time effects [2,12], (c) generalize to new constructions, and (d) encode prior knowledge from earlier sentences in a multiple sentence discourse [4,5]
We address these questions in the computational framework of reservoir computing
Summary
One of the most remarkable aspects of language processing is the rapidity with which it takes place This is revealed perhaps most clearly in event related brain potential (ERP) studies in which a word that violates predictions about the developing meaning or grammatical structure can yield brain responses to that word as rapidly as 200–600 ms [1,2,3]. Information that is provided by a sentence earlier in the discourse can cause an on-line conflict response to a word that occurs several sentences later in the discourse [3,4,5] This suggests that the brain is accumulating evidence on-line in real-time, and predicting or generating expectations about the subsequent structure of the incoming sentence. It indicates that the brain is making that accumulated knowledge available in real-time, and that it is continuously revising its predictions based on the interaction between incoming information, and the context formed by earlier inputs
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
