Iterative learning control (ILC) has been developed for decades and is mainly used to solve the repetitive control tasks. However, in the actual operation of systems, there are many non-strictly repetitive or iteration-varying factors, such as the iteration-varying reference trajectory, non-repetitive system parameters, iteration-related initial states, iteration-dependent input and output disturbances, etc. In order to solve the non-strictly repetitive problems, the High-Order Internal Model (HOIM)-based ILC is proposed. HOIM can be formulated as a polynomial in the iteration domain, which is auto-regressive. HOIM-based ILC for nonlinear systems is more complex than HOIM-based ILC for linear systems, and when the system parameters change iteratively, the Lyapunov-based analysis method is used instead of traditional contraction mapping method. Not only can HOIM be integrated into the traditional ILC, it can be also combined with other control methods, such as adaptive control, terminal control, repetitive control and so on. In this paper, we review the advances in HOIM-based ILC, systematically sort out the development and main contents of HOIM, summarize its main applications and extensions, and finally put forward some further development directions.