Oil spill accidents have gradually increased due to the continuous development of marine transportation and petroleum processing industries. Monitoring and managing marine oil spills present important economic, social, and practical implications in preventing offshore oil pollution and maintaining ecological balance. Unmanned aerial vehicle (UAV) has become a suitable carrier for low-altitude oil spill detection because of their fast deployment and low cost. Thermal infrared remote sensing images are used as the research object in this study. A method around histogram of gradient (HOG) features combined with a support vector machine (SVM) is proposed for identifying oil spills at sea to improve the accuracy of offshore low-altitude oil spill recognition and realize all-weather monitoring of offshore oil spills in offshore waters. Steps for extracting HOG features and basic principles of the SVM classification are first investigated. Image preprocessing is then performed on collected thermal infrared image data to produce samples. HOG features of samples are extracted, and the radial basis function is selected as the kernel function for training the SVM classifier. HOG features of the infrared image to be tested are calculated and then sent to the classifier for identifying the oil spills. In addition, the proposed method is compared with the back propagation(BP) neural network method and local binary pattern (LBP) combined with the SVM classification method for analysis. The results show that the oil film recognition method based on the HOG feature and SVM has a recognition accuracy of 91.3% in the environment of small infrared oil film samples, which is significantly better than the BP and LBP-SVM recognition methods, and obtains a shorter training time. The method proposed in this study has obvious advantages in terms of small sample size and processing efficiency, can meet the requirements of all-weather inspection of oil film pollutants by UAV in offshore port areas, and has great application potential in the field of maritime supervision informatization in the future.