Non-orthogonal multiple access (NOMA) is regarded as a promising technology to provide high spectral efficiency and support massive connectivity in 5G systems. Traditionally, NOMA user grouping is non-overlapping, leading to a waste of power resources within each NOMA group. Motivated by this, in this paper we propose a novel generalized user grouping (GuG) concept for NOMA from an overlapping perspective, which allows each user to participate in multiple user groups but subject to individual maximum power constraint. In order to achieve effective GuG and maximize the system sum rate, we formulate a joint power control and GuG optimization problem. Then we further provide a machine learning-based GuG scheme to obtain the optimized feasible GuG and the optimal power control solutions efficiently, in which the established machine learning-based model is exploited to explore the relative relationships of channel gains of users and obtain several fixed grouping patterns via Merge operation. Simulation results verify the efficiency of GuG in NOMA systems and indicate that compared with traditional NOMA user grouping schemes, our proposed GuG scheme achieves significant performance gains in terms of system sum rate.