The right posterolateral portions of the cerebellum (crus-I/II) are involved in language processing. However, their functional role in language remains unknown. The cerebellum is hypothesized to acquire an internal model that is a functional copy of mental representations in the cerebrum and to contribute to cognitive function. In this research, based on the cerebellar internal model hypothesis, we conducted task-based and resting-state functional magnetic resonance imaging (fMRI) experiments to investigate the role of the cerebellum in the syntactic and semantic aspects of comprehension of sentences. In a syntactic task, participants read sentences with center-embedded hierarchical structures. The hierarchical level-dependent activity was found in the right crus-I as well as Broca's area (p < 0.05, voxel-based small volume correction (SVC)). In a semantic task, the participants read three types of sentences for investigation of sentence-level, phrase-level, and word-level semantic processing. The semantic level-dependent activity was found in the right crus-II as well as in the left anterior temporal lobe and the left angular gyrus (p < 0.05, voxel-based SVC). Moreover, the right crus-I/II showed significant activity when the cognitive load was high. Resting-state fMRI demonstrated intrinsic functional connectivity between the right crus-I/II and language-related regions in the left cerebrum (p < 0.05, voxel-based SVC). These findings suggest that the right crus-I and crus-II are involved, respectively, in the syntactic and semantic aspects of sentence processing. The cerebellum assists processing of language in the cerebrum when the cognitive load is high.