Recommender systems are widely used in e-commerce, movies, music, social media, and other fields because of their personalized recommendation functions. The recommendation algorithm is used to capture user preferences, item characteristics, and the items that users are interested in are recommended to users. Matrix factorization is widely used in collaborative filtering algorithms because of its simplicity and efficiency. However, the simple dot-product method cannot establish a nonlinear relationship between user latent features and item latent features or make full use of their personalized information. The model of a neural network combined with an attention mechanism can effectively establish a nonlinear relationship between the potential features of users and items and improve the recommendation accuracy of the model. However, it is difficult for the general attention mechanism algorithm to solve the problem of attention interaction when the number of features between the users and items is not the same. To solve the above problems, this paper proposes an attention interaction matrix factorization (AIMF) model. The AIMF model adopts a symmetric structure using MLP calculation. This structure can simultaneously extract the nonlinear features of user latent features and item latent features, thus reducing the computation time of the model. In addition, an improved attention algorithm named slide-attention is included in the model. The algorithm uses the sliding query method to obtain the user’s attention to the latent features of the item and solves the interaction problem among different dimensions of the user, and the latent features of the item.