Active large scale surveillance of indoor and outdoor environments with multiple cameras is becoming an undeniable necessity in today's connected world. Enhanced computational and storage capabilities in smart cameras establish them as promising platforms for implementing intelligent and autonomous surveillance networks. However, poor resolution, limited number of samples per object, and pose variation in multi-view surveillance streams, make the task of efficient image representation highly challenging. To address these issues, we propose an efficient and powerful convolutional neural network (CNN) based framework for features extraction using embedded processing on smart cameras. Efficient, high performance, pre-trained CNNs are separately fine-tuned on persons and vehicles to obtain discriminative, low dimensional features from segmented surveillance objects. Furthermore, multi-view queries of surveillance objects are used to improve retrieval performance. Experiments reveal better efficiency and retrieval performance in different surveillance datasets.