Problems such as low light, similar background colors, and noisy image acquisition often occur when collecting images of lunar surface obstacles. Given these problems, this study focuses on the AD-Census algorithm. In the original Census algorithm, in the bit string calculated with the central pixel point, the bit string will be affected by the noise that the central point is subjected to. The effect of noise results in errors and mismatching. We introduce an improved algorithm to calculate the average window pixel for solving the problem of being susceptible to the central pixel value and improve the accuracy of the algorithm. Experiments have proven that the object contour in the grayscale map of disparity obtained by the improved algorithm is more apparent, and the edge part of the image is significantly improved, which is more in line with the real scene. In addition, because the traditional Census algorithm matches the window size in a fixed rectangle, it is difficult to obtain a suitable window in the image range of different textures, affecting the timeliness of the algorithm. An improvement idea of area growth adaptive window matching is proposed. The improved Census algorithm is applied to the AD-Census algorithm. The results show that the improved AD-Census algorithm has been shown to have an average run time of 5.3% and better matching compared to the traditional AD-Census algorithm for all tested image sets. Finally, the improved algorithm is applied to the simulation environment, and the experimental results show that the obstacles in the image can be effectively detected. The improved algorithm has important practical application value and is important to improve the feasibility and reliability of obstacle detection in lunar exploration projects.