For infrastructure health monitoring, defects in concrete are periodically inspected by hammering test. Conventional hammering inspection is manually conducted to detect internal concrete defects by recognizing the sounds generated from the hammer strike. Therefore, a robot-based hammering inspection system can realize the inspection automatically. However, those robots have several elements such as noise-resistant acoustic analysis, weight reduction of the robot, constant striking mechanism, and stable robot movement; so integrated research has not been achieved. This paper introduces a hammering inspection system using robot that solves these problems. In the proposed system, a multi-copter type mobile robot realizes a stable attitude on a structure using thrust force to press the robot body onto the structure surface, while a lightweight hammering mechanism constantly strikes a wall. During inspection, noises from multi-copter affect acoustic analysis. Therefore, the proposed method clarifies the features of hammering sound and propeller noise and analyzes hammering sounds. This developed hammering inspection system is the first to realize all functions required for comprehensive hammering inspection, and its accuracy to detect concrete defects from hammering sounds was 81.7% through the experiments.