The paper focuses on the problem of a monochromatic terahertz TE-polarized wave propagation in a plane dielectric layer filled with a homogeneous isotropic medium; one of the boundaries of the waveguide is covered with a layer of graphene. In fact, the paper aims to find the eigenwaves of the described waveguiding structure. On the one hand, in the study, energy losses both in the dielectric layer and in the graphene layer are neglected; the latter assumption is reasonable in the terahertz range of electromagnetic radiation (on which the paper focuses), where graphene has a strong plasmonic response and much less loss. On the other hand, this study takes into account the significant third-order nonlinearity resulting from the interaction of the electromagnetic wave with the charge carriers in the graphene layer. The paper aims to study the guiding properties of the above structure using primarily an analytical approach. The wave propagation problem is reduced to an eigenvalue problem, where one of the boundary conditions is nonlinear with respect to the sought-for function. The main result of the paper is a dispersion equation allowing for a waveguide of a given thickness to determine a set of its propagation constants and, consequently, a set of its eigenwaves. It is worth noting that the dispersion equation being written in an explicit form can be used to obtain deep qualitative results related to the solvability of the problem and the properties of its solutions. For example, in the paper, the existence of several propagation constants (and, consequently, the eigenwaves) of the studied waveguiding structure is proved under some conditions. Besides studying the problem analytically, the paper presents some numerical results as well. In particular, the presented figures demonstrate how the nonlinearity in graphene affects the propagation constants and eigenwaves, providing the dispersion curves and eigenwaves for nonlinear graphene as well as for the linear one.