Graph-based intelligent systems are emerging in the field of transportation systems. Knowledge graphs help to provide semantic and interconnectivity capabilities to the intelligent transportation system. In this paper, we propose a graph-based method for detecting outliers in the trajectory. Normal and outlier graphs are constructed using directed weighted graphs. Then, comparison with source and target graphs leading to vectors is performed. The similarity measure is used, which measures common nodes and edges. The features are then used by the machine learning based algorithm to classify the trajectory. Instead of a manually tuned parameter, the tree-based pipeline optimization method selects the best classifier and its hyperparameter. Then, the tuned model is compared with the traditional algorithms, i.e., random forest, decision tree, Naïve Bayes, and KNN. To evaluate the system under real conditions, an experiment is performed on a real dataset of trajectories. The results show that the graph-based method performs well and achieves an F-value of 0.81, while the optimization model achieves an F-value of 0.87. The graph-based model matched with the learning method helps to detect outliers and deviation points of the trajectory with high precision.