Building material enterprises (BMEs) and waste recycling enterprises (WREs) are two critical stakeholders in construction waste recycling, and their decisions toward waste recycling are influential to development of waste recycling market. However, the present literature fails to consider consumers’ quality perceptions of recycled materials when investigating stakeholder decisions in the market, which significantly affects understanding of stakeholder decisions on waste recycling. To address the research gap, this study investigates from a theoretical perspective the optimal strategies of the BME and the WRE by considering consumers’ quality perceptions toward recycled materials. The game theory is employed to develop models and analyze stakeholder decisions in the context of different construction waste (CW) recycling scenarios. It is found that the consideration of consumers’ quality perceptions toward recycled materials shapes three CW recycling scenarios (i.e., business stripping mode, business monopoly mode, and business cooperation mode) in the waste recycling markets. When there is a demand for recycled materials, the BME should take good use of its labor and capital advantages to actively develop the waste recycling market for higher profits. However, when there are already independent WREs, the BME should perform waste recycling business jointly with the WRE. In addition, there exists stable competitive market equilibrium, depending on low consumer quality perception of recycled materials produced by the BME and small consumer quality perception disparity between recycled materials provided by the two enterprise groups. Under the circumstance, the BME and the WRE can achieve a win-win situation for gaining profits. This theoretical study contributes to a new perspective (i.e., consumers’ quality perceptions) for better understanding decisions of major stakeholders (i.e., the BME and the WRE) on CW recycling. In addition, the study for the first time introduces the game theory into explaining CW recycling, which may enhance the effectiveness of policies aiming at promoting CW recycling practices.