With popularity of mass customization, enterprises urgently need to improve reconfigurability to handle rapidly changing market demands. Reconfigurable machines have been widely applied in the flexible assembly job shop. These reconfigurable machines can be equipped with various and limited auxiliary modules (AMs) to provide multiple alternative functions to manufacture numerous products. Therefore, this study addresses flexible assembly job shop scheduling problem considering reconfigurable machines with limited AMs to minimize makespan. First, a mixed-integer linear programming (MILP) model is formulated to define the problem. Then, a cooperative co-evolutionary matheuristic algorithm (CCMA) is presented to efficiently tackle large-scale FAJSP-RM problem. In this algorithm, a MILP-based evolution method, which relies on a decomposed MILP (D-MILP) with known sequence decisions, is proposed to explore the optimal machine and AM assignments. Meanwhile, a collaborative initialization and dual collaborative strategy are proposed to improve the performance of the proposed CCMA. Comprehensive experiments are conducted on 720 instances, extended from the well-known Fdata and BRdata benchmarks, to evaluate the proposed MILP model and CCMA algorithm. The results illustrate that the MILP model can produce optimal solutions for small-scale instances. The MILP-based evolution method improves the performance of the CCMA algorithm by 12.63, and the CCMA outperforms other well-known algorithms from numerical, statistical, differential and stable analysis.