Estimation of traffic flow parameters based on computer vision is still a very popular challenge. In particular, the detection problems such as occlusion, small targets, luminance variations, and lighting jitter, etc., are caused by dense traffic scenes and complex weather conditions. Previous studies about detecting and tracking methods focused on daytime and nighttime or single weather condition, but the performance of this model will decline dramatically due to complex conditions. In this paper, a framework for estimating traffic flow parameters in complex environments is presented, in which Ghost-YOLOv8 vehicle detection model is proposed. GhostConv is used to replace part of Conv and a new C2fGhost module is designed to replace part of C2f, reducing the number of parameters and improving the detection performance of the model; The global attention mechanism module is added to the neck network, which strengthens the semantic and location information in the features and improves the model’s ability of feature fusion; In view of the loss of semantic information caused by different scales in detecting small targets, a small target detection layer is added to enhance a combination of deep and shallow semantic information; The GIoU bounding loss function is used instead of the original loss function, which improves the performance of the network’s bounding box regression. Then, combining DeepSORT tracking algorithm and virtual line counting method, the proposed framework estimates traffic flow parameters, including volume, speed, and density in complex environments. The experimental results show that compared with the original model in the UA-DETRAC dataset, the improved YOLOv8 model’s the accuracy is increased by 1.4%, and the parameter number and model size are reduced by 0.229G and 0.2MB respectively. In addition, the framework has good robustness, reaching 97.56% accuracy when estimating average traffic flow parameters. It is sufficient to overcome complex weather conditions and effectively help control traffic for intelligent transportation systems and traffic management to provide good data support. In conclusion, it shows that the model can reduce the number and size of model parameters and improve the detection precision as well as meeting the requirements of edge computing devices and having better real-time performance, so it has practical application value.