Fog and haze (F-H) weather has been occurring frequently in China since 2012, which affects the output power of photovoltaic (PV) generation dramatically by directly weakening solar irradiance and aggravating dust deposition on PV panels. The ultra-short-term forecast method presented in this study would help to fully reflect the dual effects of F-H on PV output power. Aiming at the weakening effect on solar irradiance, estimation models of atmospheric aerosol optical depth (AOD) based on particle matter (PM) concentration were established with machine learning (ML) method, and the total irradiance received by PV panels was calculated based on simplified REST2 model. Aiming at the aggravating effect on dust deposition on PV panels, sample set of “cumulative PM concentration—efficiency reduction” was constructed through special measurement experiments, then the efficiency reduction under certain dust deposition state was estimated with similar-day choosing method. Based on photoelectric conversion model, PM concentration prediction and weather forecast information, ultra-short-term forecast of PV output power was realized. Experimental results proved the validity and feasibility of the presented forecast method.