The semiconductor back end manufacturing process is a complicated process including many stages. This manufacturing process is labor intensive and time consuming. To improve the operation process specially in reducing the total inspection time and cost, this study proposes a multi-stage prediction and classification model to make a decision whether a chip or die or wafer needs to be checked before moving to the next process or stage. In particular, Back propagation neural network (BPNN) models are designed for several main stages of inspection based on the historical data. The proposed approach was validated based on the historical data from the leading semiconductor back end manufacturing factory in Vietnam. Then, the proposed model was implemented in the factory for one million chips of 50 thousands wafers. Compared with the traditional approach, the proposed approach reduced up to 73 percent of the total average inspection time of the die attach, die attach cure, wire bonding, molding, post mold cure, lead finishing, and trimming stages. The model assures high accuracy throughout the manufacturing process.