This paper presents an effective traffic video surveillance system for detecting moving vehicles in traffic scenes. Moving vehicle identification process on streets is utilized for vehicle tracking, counts, normal speed of every individual vehicle, movement examination, and vehicle classifying targets and might be executed under various situations. In this paper, we develop a novel hybridization of artificial neural network (ANN) and oppositional gravitational search optimization algorithm (ANN–OGSA)-based moving vehicle detection (MVD) system. The proposed system consists of two main phases such as background generation and vehicle detection. Here, at first, we develop an efficient method to generate the background. After the background generation, we detect the moving vehicle using the ANN–OGSA model. To increase the performance of the ANN classifier, we optimally select the weight value using the OGSA algorithm. To prove the effectiveness of the system, we have compared our proposed algorithm with different algorithms and utilized three types of videos for experimental analysis. The precision of the proposed ANN–OGSA method has been improved over 3% and 6% than the existing GSA-ANN and ANN, respectively. Similarly, the GSA-ANN-based MVD system attained the maximum recall of 89%, 91%, and 91% for video 1, video 2, and video 3, respectively.