In recent years, due to the rise of the concept of intelligent ship and the development of the intelligent ship, the ship intelligent has become a global shipping the trend of The Times, with the rapid development of maritime shipping industry, sea lanes and port ship intensity increasing, shipping safety risk increasingly improve, intelligent traffic regulation becomes increasingly important task, shipping information acquisition is more accurate. Ship tracking and identification system is not only the core of shipping safety supervision system, but also has an effective role in the collection of ship navigation information and intelligent supervision of navigation traffic. Computer vision is a technology developed rapidly in recent years, as the current research hotspot, with its access to large amount of information, and the advantages of fast response speed, strong anti-jamming capability compared with traditional of AIS and radar easily affected by obstacles, affected by the operator subjective behavior of communication system device, it can well realize the target ship recognition and tracking, to the modern channel intelligent regulation of ship sailing information collection, transportation, ship navigation safety are very important significance. This article mainly from the principle of development background significance, system characteristics, the visual technology, tracking algorithm for visual identification and tracking system to do a detailed description of principle, on the basis of AIS and radar, based on traditional visual algorithm combined with some data and AIS system on personal improvement ideas on the application of the system innovation, and finally puts forward personal has been developed out of a visual tracking innovative device of the ship.