Support Vector Machine (SVM) is often used in regression and classification problems. However, SVM needs to find proper kernel function to solve high-dimensional problems. We propose an improved Sparrow Search Algorithm Support Vector Machine (ISSA-SVM) algorithm to optimize the SVM kernel parameters. First, the problem of slow convergence due to the lack of ergodicity and poor diversity of the initial population is effectively overcome by using Sine chaotic map. Second, adaptive dynamic weight factors are induced not only to balance the global and local search capabilities, but also accelerate the convergence speed of the sparrow search algorithm. The simulation results of 11 benchmark test functions show that ISSA has faster convergence, more accurate search capability, and easier to jump out of local extremes than the SSA, Gray Wolf Optimization (GWO) and Whale Optimization Algorithm (WOA). That indicates ISSA has better convergence, better robustness, and stronger competitiveness. The experimental results on the coal gangue dataset show that the classification accuracy of ISSA-SVM algorithm is improved by 7.09% and 4.25% compared with SVM and SSA-SVM, respectively. Meanwhile, the classification time for a single image frame of ISSA-SVM algorithm is reduced by 20.15% and 13.74% compared with SVM and SSA-SVM, respectively.