The usage of automobiles in cities and metropolitan areas has increased drastically throughout the years and there is a need to monitor the flow of road traffic to improve the traffic congestion and safety. One of the best ways to monitor the traffic is using an artificial intelligence and machine learning. An automatic vehicle tracking system based on artificial intelligence and machine learning can offers capability to analyse the real-time traffic video data for the purpose of traffic surveillance. The computer vision is one of the subsets in machine learning that can train the computer to understand the visual data and perform specific tasks such as object detection and classification. A Vision-based system can be proposed to detect road accidents, predict traffic congestion and further road traffic analytics. This can improve the safety in transportation where it can recognize types of vehicles on the road, detecting road accidents, predicting the traffic congestion and further road traffic analytics. In the context of road traffic monitoring, the parameters of the traffic such as the type and number of vehicles that passes through must be recorded in order to gain valuable insights and make prediction such as the occurrence of traffic congestion. However, this requires reliable informative and accurate data as input for analytics. Therefore, in this research the deep learning model for vehicle classification is investigated to detect, classify types of vehicles and further predictive analytics. The vehicle classification is proposed based on Single Shot Detector (SSD) architecture model. The proposed model is tested on five different classes of vehicles with a total of 1263 images. Experimental results show that SSD model able to achieve 0.721 of precision, 0.741 of recall and 0.731 of F1 Score. Finally, the result show that the SSD model is more accurate among all the models for all the performance measure with the difference of more than 0.052 of precision, 0.706 of recall and 0.05 of F1 Score.