This paper describes an object tracking system using an autonomous Micro Air Vehicle (MAV) and demonstrate its potential use for civilian purposes. The vision-based control system relies on color and feature based vision algorithm for target detection and tracking, Kalman filters for relative pose estimation, and a nonlinear controller for MAV stabilization and guidance. The color based vision algorithm uses simple and fast technique object color detection using integral images, while optical flow is utilized for feature tracking. Each vision algorithm has its own weakness depending on the environment when vision-based tracking is done. The color based object tracking very much depends on suitable light condition, and the tracked object and its background color. The feature based object tracking works well with object with strong features but risks loss of detection when there is noise in the streaming image while tracking. An algorithm that combines the two approaches is presented to compensate each single vision algorithm weakness. The vision algorithm relies on information from a single on-board camera. An arbitrary target can be selected in real-time from the ground control station, thereby outperforming template and learning-based approaches. Experimental results obtained from outdoor (light tests, showed that the vision-control system enabled the MAV to track and hover above the target as long as the battery is available. We also propose this object tracking system to track a moving object on the ground as well in leader-following system setup.