Due to the complexity of the environment and the uncertainty of the target characteristics, the existing photoelectric detection imaging system has a great difference in the target acquisition capability, improving the detection capability of the photoelectric detection imaging equipment focuses on the detection probability of the optical equipment. According to the detection principle of photoelectric detection imaging system, we study the target detection probability calculation model of photoelectric detection imaging system from the perspective of signal to noise ratio and detection distance, and analyze the detection probability calculation method under different signal to noise ratio. Using the gray value of the target and the gray function of the complex background detected by the optical detection imaging system, the target detection probability functions corresponding to different target gray values and different background radiance are derived. Through calculation and experimental analysis, the results show that the target detection probability calculation method can effectively evaluate the detection probability of photoelectric detection imaging system under complex background in practical application.