Object detection and recognition technology plays an important role in video advertising. Based on the deep learning algorithm, this paper first builds a real-time object detection model (Yes-Net) based on image global information which combines the advantages of CNN and RNN algorithm together and has achieved significant effects compared with other object detection algorithms. We used this model to build an accurate video advertising framework and applied the framework in an IPTV platform of a telecommunication operator to collect comparative data. We first performed image detection and recognition in video playing, and then associated the recognition results with the ads database to conduct real-time classification and accurate ads delivery. It proves that compared with traditional advertisement delivery method, this new framework dramatically improves the click-through rate, and achieves better results.