Chemical mechanical polishing/planarization (CMP) of wafers is routinely used as the last mechanical process to produce super-smooth or critically planarized surfaces by removing machining-induced surface/subsurface damages and at the same time reduce the surface roughness to the level nanometers or even sub-nanometers without sacrificing the overall flatness. Successful CMP of monocrystalline silicon carbide (SiC) and sapphire substrates—two major substrates besides silicon for fabrication of high-brightness light emission diode (LED) devices, and also two important optical materials for space applications—demands a deeper mechanistic understanding into the CMP material removal process of these two materials. One of crucial challenges is to reveal the mechanism of formation of the chemical reaction layer affected by chemical-mechanical interaction during CMP. This paper critically reviews the current status of relevant progresses, controversies and challenges. First, we reviewed the debate about the thermodynamic behavior of formation of the chemical reaction layer by carefully comparing the original experimental results and arguments by different groups, including numerous studies on the equilibrium of the reactions involved and the stable reaction products. Second, the inconsistencies on the reaction kinetic behavior of formation and removal of the reaction layer, i.e., the temperature dependent materials removal rates in terms of activation energy, were critically compared and re-analyzed by deriving the numerical values of activation energy from the existing literature. Third, the challenge in establishing the model CMP substrate systems was highlightened in order to stress the urgent requirement for a firm foundation without which the results of different labs were difficult to be critically compared. Last, problems of incomplete characterization of the structure and compositions of the reaction layer and substrates as well as the poor cross-lab repeatability of characterization results were discussed with examples. In the perspective part, we propose a few directions for possible breakthroughs in the area of the mechanisms of formation of the chemical reaction layer: (1) establishment of reliable model CMP substrate systems with controlled surface structure by taking into account a few key parameters including surface/ subsurface defect density, misorientation angle (off-cut angle) as well as crystallography planes exposed; (2) determination and control of the reaction rate to facilitate decoupling reaction and diffusion processes with the help of the frictional force microscopy technique. A multidisciplinary strategy for addressing the issues discussed above is thus strongly advocated through combining materials/surface science/chemistry related approaches—careful and detailed characterization using a variety of state-of-the-art analytical techniques, precise control of surface structure and defects for developing model CMP substrate systems, in-depth analyses of surface structure and compositions, chemical thermodynamic and kinetic studies on chemical reaction mechanisms, as well as microscopic frictional/wear tests and surface force measurements. This review will also have important implications for CMP processing of many more different types of materials other than sapphire and SiC, including but not limited to other promising LED substrates such as GaN and ZnO, optical materials for laser inertial confinement fusion applications such as hard-brittle fused silica and soft-brittle KDP crystals, as well as conventional and novel ceramics and crystals for strong lasers such as YAG and sesquioxides (Lu2O3, Y2O3).