In order to prevent the danger caused by falling tiles from high-rise buildings, a rapid and effective non-destructive testing and evaluation (NDT&E) technique has been developed to assess the tile–wall bonding quality. The proposed technique is based on sounds excited by controlled impacts, and can easily be integrated with climbing robots to automate the tile–wall inspection process without the need for human workers to work at life-threatening height. To facilitate the result evaluation and maintenance planning, the approximate size of defective area needs to be assessed and determined. Though it is well-known that the natural frequency of flexural vibration is related to the size of the debonded area, strong multiple-mode frequencies caused by complex shapes of the defects or impacting at geometric edge (not at the center) of the defects impose significant difficulties in extracting automatically the natural frequency from the sound signals. In this paper, a fuzzy scheme is introduced to improve the robustness and accuracy of defects assessment. Based on fuzzy theory, vibration principle and human experience, a fuzzy logic model relating the characteristics of impact sounds to the approximate size of the defect is developed and utilized. The design of the fuzzy system, including membership functions and fuzzy reasoning rules, is also provided. To demonstrate the validity of the proposed method, experimental results obtained from physical tile walls are presented and discussed.