Dense Scale Invariant Feature Transform Research Articles

Binary Dense SIFT Flow Based Position-Information Added Two-Stream CNN for Pedestrian Action Recognition

Pedestrian behavior recognition in the driving environment is an important technology to prevent pedestrian accidents by predicting the next movement. It is necessary to recognize current pedestrian behavior to predict future pedestrian behavior. However, many studies have recognized human visible characteristics such as face, body parts or clothes, but few have recognized pedestrian behavior. It is challenging to recognize pedestrian behavior in the driving environment due to the changes in the camera field of view due to the illumination conditions in outdoor environments and vehicle movement. In this paper, to predict pedestrian behavior, we introduce a position-information added two-stream convolutional neural network (CNN) with multi task learning that is robust to the limited conditions of the outdoor driving environment. The conventional two-stream CNN is the most widely used model for human-action recognition. However, the conventional two-stream CNN based on optical flow has limitations regarding pedestrian behavior recognition in a moving vehicle because of the assumptions of brightness constancy and piecewise smoothness. To solve this problem for a moving vehicle, the binary descriptor dense scale-invariant feature transform (SIFT) flow, a feature-based matching algorithm, is robust in moving-pedestrian behavior recognition, such as walking and standing, in a moving vehicle. However, recognizing cross attributes, such as crossing or not crossing the street, is challenging using the binary descriptor dense SIFT flow because people who cross the road or not act the same walking action, but their location on the image is different. Therefore, pedestrian position information should be added to the conventional binary descriptor dense SIFT flow two-stream CNN. Thus, learning biased toward action attributes is evenly learned across action and cross attributes. In addition, YOLO detection and the Siamese tracker are used instead of the ground-truth boundary box to prove the robustness in the action- and cross-attribute recognition from a moving vehicle. The JAAD and PIE datasets were used for training, and only the JAAD dataset was used as a testing dataset for comparison with other state-of-the-art research on multitask and single-task learning.

Segmentation of pancreatic ductal adenocarcinoma (PDAC) and surrounding vessels in CT images using deep convolutional neural networks and texture descriptors

Fully automated and volumetric segmentation of critical tumors may play a crucial role in diagnosis and surgical planning. One of the most challenging tumor segmentation tasks is localization of pancreatic ductal adenocarcinoma (PDAC). Exclusive application of conventional methods does not appear promising. Deep learning approaches has achieved great success in the computer aided diagnosis, especially in biomedical image segmentation. This paper introduces a framework based on convolutional neural network (CNN) for segmentation of PDAC mass and surrounding vessels in CT images by incorporating powerful classic features, as well. First, a 3D-CNN architecture is used to localize the pancreas region from the whole CT volume using 3D Local Binary Pattern (LBP) map of the original image. Segmentation of PDAC mass is subsequently performed using 2D attention U-Net and Texture Attention U-Net (TAU-Net). TAU-Net is introduced by fusion of dense Scale-Invariant Feature Transform (SIFT) and LBP descriptors into the attention U-Net. An ensemble model is then used to cumulate the advantages of both networks using a 3D-CNN. In addition, to reduce the effects of imbalanced data, a multi-objective loss function is proposed as a weighted combination of three classic losses including Generalized Dice Loss (GDL), Weighted Pixel-Wise Cross Entropy loss (WPCE) and boundary loss. Due to insufficient sample size for vessel segmentation, we used the above-mentioned pre-trained networks and fine-tuned them. Experimental results show that the proposed method improves the Dice score for PDAC mass segmentation in portal-venous phase by 7.52% compared to state-of-the-art methods in term of DSC. Besides, three dimensional visualization of the tumor and surrounding vessels can facilitate the evaluation of PDAC treatment response.

Motor Fault Diagnosis Using Image Visual Information and Bag of Words Model

In order to solve the tough issue that is difficult to extract the representative fault features of the hybrid vibration signals from the various industrial applications, a motor fault diagnosis method based on image visual information and bag of words model (BoW) are proposed in this paper. Based on the redundancy information of raw signals, the mapping which is between the original fault signal and the visual information is obtained by the methods of SDP (Symmetrized Dot Pattern) image and dense SIFT (Scale Invariant Feature Transform) features. The bag of visual word (BoVW) and pyramid histogram intersection kernel support vector machine (PHIK-SVM) are applying to complete the fault diagnosis and state recognition of related motors. The test results reveal that the diagnostic accuracy of four fault states could reach 99.50%, moreover, the diagnostic accuracy of different operating conditions under each fault could reach 91.83% as well. Compared with the traditional methods, the proposed method is able to obtain the detailed characteristics of various motors from the perspective of visual knowledge, furthermore, it is more robust and without complex data processing.

A new region-of-interest (ROI) detection method using the chan-vese algorithm for lung nodule classification

Suspicious regions in chest x-rays are detected automatically, and these regions are classified into three types, including “malignant nodule”, “benign nodule”, and “no nodule” in this study. Firstly, the areas except the lung tissues are removed in each chest x-ray using the thresholding method. Then, Poisson noise was removed from the images by applying the gradient filter. Ribs may overlap onto nodules. Since this circumstance will make the detection of a nodule difficult, it is necessary to distinguish and suppress the ribs. The location of the rib bones is determined by a template matching method, and then the corresponding bones are suppressed by applying the Gabor filter. After this stage, suspicious tissues in the chest x-rays are specified using the Chan-Vese active contour without edges. Then, some features are extracted from these suspicious regions. Six different features are extracted: Statistical, Histogram of Oriented Gradients (HOG)-based, Local Binary Pattern (LBP)-based, Geometrical, Gray Level Co-Occurrence Matrix (GLCM) Texture-based and Dense Scale Invariant Feature Transform (DSIFT)-based. Then, the classification stage is achieved using these features. The best classification result is obtained using statistical, LBP-based, and HOG-Based features. The classification results are evaluated with sensitivity, accuracy, and specificity analyses. K-Nearest Neighbour (KNN), Decision Tree (DT), Random Forest (RF), Logistic Linear Classifier (LLC), Support Vector Machines (SVM), Fisher’s Linear Discriminant Analysis (FLDA), and Naive Bayes (NB) methods are used for the classification purpose separately. The random forest classifier gives the best results with 57% sensitivity, 66% accuracy, 81% specificity values.

Classification of Breast Cancer using Histology images: Handcrafted and Pre-Trained Features Based Approach

Breast cancer has become a critical disease in women. The number of patients with breast cancer is quite high in India. It is of paramount importance to detect the disease in advance. Digital histopathology is one of the most advanced techniques for detection using machine learning. Artificial intelligence is going to be like a sunrise in the field of medicine. Deep neural networks have been successfully applied to the problem under consideration in the past. As, we know the feature extraction is one of the essential and crucial steps in case of classification. In this paper, we compare two approaches, first is feature extraction using traditional Handcrafted based and other is Transfer Learning based model (Pre-trained) for multiclass classification of Breast Cancer using Convolutional Neural Network (CNN) as a classifier. The models are trained using handcrafted features like Seeped Up Robust Features (SURF) and Dense Scale Invariant Feature Transform (DSIFT) techniques, later these extracted features are encoded by Locality Constrained Linear Coding method (LLC). In pre-trained model we have used VGG16, VGG19, ResNet50, GoogLeNet for feature extraction. The maximum accuracy for “SURF+CNN” is 92.88% for Handcrafted feature and in case of Pre-trained “GoogLeNet+ CNN” model gives 94%, both for 400X magnification factor.

Bone age assessment of iranian children in an automatic manner

Background:Bone age assessment (BAA) is a radiological process with the aim of identifying growth disorders in children. The objective of this study is to assess the bone age of Iranian children in an automatic manner.Methods:In this context, three computer vision techniques including histogram of oriented gradients (HOG), local binary pattern (LBP), and scale-invariant feature transform (SIFT) are applied to extract appropriate features from the carpal and epiphyseal regions of interest. Two different datasets are applied here: the University of Southern California hand atlas for training this computer-aided diagnosis (CAD) system and Iranian radiographs for evaluating the performance of this system for BAA of Iranian children. In this study, the concatenation of HOG, LBP, and dense SIFT feature vectors and background subtraction are applied to improve the performance of this approach. Support vector machine (SVM) and K-nearest neighbor are used here for classification and the better results yielded by SVM.Results:The accuracy of female radiographs is 90% and of male is 71.42%. The mean absolute error is 0.16 and 0.42 years for female and male test radiographs, respectively. Cohen's kappa coefficients are 0.86 and 0.6, P < 0.05, for female and male radiographs, respectively. The results indicate that this proposed approach is in substantial agreement with the bone age reported by the experienced radiologist.Conclusion:This approach is easy to implement and reliable, thus qualified for CAD and automatic BAA of Iranian children.

Automated detection of diabetic retinopathy in fundus images using fused features.

Diabetic retinopathy (DR) is one of the severe eye conditions due to diabetes complication which can lead to vision loss if left untreated. In this paper, a computationally simple, yet very effective, DR detection method is proposed. First, a segmentation independent two-stage preprocessing based technique is proposed which can effectively extract DR pathognomonic signs; both bright and red lesions, and blood vessels from the eye fundus image. Then, the performance of Local Binary Patterns (LBP), Local Ternary Patterns (LTP), Dense Scale-Invariant Feature Transform (DSIFT) and Histogram of Oriented Gradients (HOG) as a feature descriptor for fundus images, is thoroughly analyzed. SVM kernel-based classifiers are trained and tested, using a 5-fold cross-validation scheme, on both newly acquired fundus image database from the local hospital and combined database created from the open-sourced available databases. The classification accuracy of 96.6% with 0.964 sensitivity and 0.969 specificity is achieved using a Cubic SVM classifier with LBP and LTP fused features for the local database. More importantly, in out-of-sample testing on the combined database, the model gives an accuracy of 95.21% with a sensitivity of 0.970 and specificity of 0.932. This indicates the proposed model is very well-fitted and generalized which is further corroborated by the presented train-test curves.

Human Visual Perception-Based Multi-Exposure Fusion Image Quality Assessment

Compared with ordinary single exposure images, multi-exposure fusion (MEF) images are prone to color imbalance, detail information loss and abnormal exposure in the process of combining multiple images with different exposure levels. In this paper, we proposed a human visual perception-based multi-exposure fusion image quality assessment method by considering the related perceptual features (i.e., color, dense scale invariant feature transform (DSIFT) and exposure) to measure the quality degradation accurately, which is closely related to the symmetry principle in human eyes. Firstly, the L1 norm of chrominance components between fused images and the designed pseudo images with the most severe color attenuation is calculated to measure the global color degradation, and the color saturation similarity is added to eliminate the influence of color over-saturation. Secondly, a set of distorted images under different exposure levels with strong edge information of fused image is constructed through the structural transfer, thus DSIFT similarity and DSIFT saturation are computed to measure the local detail loss and enhancement, respectively. Thirdly, Gauss exposure function is used to detect the over-exposure or under-exposure areas, and the above perceptual features are aggregated with random forest to predict the final quality of fused image. Experimental results on a public MEF subjective assessment database show the superiority of the proposed method with the state-of-the-art image quality assessment models.

Classifying suspicious content in tor darknet through Semantic Attention Keypoint Filtering

One of the tasks Law Enforcement Agencies are responsible for is to find evidence of criminal activities in the Darknet. However, visiting thousands of domains to locate visual information containing illicit acts manually requires a considerable amount of time and human resources. To support this task, in this paper, we explore the automatic classification of images uploaded to Tor darknet.Unfortunately, the foreground objects on such images are not always presented standalone, without background, due to the environmental conditions. To address this challenge on the digital investigation of Tor darknet visual content, we propose to classify automatically only relevant parts of the image combining saliency maps, i.e. to select the regions with the most salient information, with Bag of Visual Words (BoVW). We introduce Semantic Attention Keypoint Filtering (SAKF), a filtering strategy that removes non-significant features at a pixel level that mainly do not belong to the object of interest or foreground. We assessed SAKF on seven publicly available datasets, obtaining from 1.64 to 15.73 points higher accuracies than the method set as the baseline, i.e. BoVW using dense SIFT (Scale-Invariant Feature Transform) descriptors. We also compared SAKF filtering performance against the deep features extracted from two well-known Convolutional Neural Network (CNN) architectures, namely MobileNet and ResNet50.Experimental results reveal the effectiveness of the proposed approach and highlight that the use of automatic image classification could be advantageous to support daily Law Enforcement Agencies investigations on Tor darknet.

Visual Tea Leaf Disease Recognition Using a Convolutional Neural Network Model

The rapid, recent development of image recognition technologies has led to the widespread use of convolutional neural networks (CNNs) in automated image classification and in the recognition of plant diseases. Aims: The aim of the present study was to develop a deep CNNs to identify tea plant disease types from leaf images. Materials: A CNNs model named LeafNet was developed with different sized feature extractor filters that automatically extract the features of tea plant diseases from images. DSIFT (dense scale-invariant feature transform) features are also extracted and used to construct a bag of visual words (BOVW) model that is then used to classify diseases via support vector machine(SVM) and multi-layer perceptron(MLP) classifiers. The performance of the three classifiers in disease recognition were then individually evaluated. Results: The LeafNet algorithm identified tea leaf diseases most accurately, with an average classification accuracy of 90.16%, while that of the SVM algorithm was 60.62% and that of the MLP algorithm was 70.77%. Conclusions: The LeafNet was clearly superior in the recognition of tea leaf diseases compared to the MLP and SVM algorithms. Consequently, the LeafNet can be used in future applications to improve the efficiency and accuracy of disease diagnoses in tea plants.

DENSE SIFT FEATURE AND LOCAL NAIVE BAYES NEAREST NEIGHBOR FOR FACE RECOGNITION

Human face recognition is a technology which is widely used in life. There have been much effort on developing face recognition algorithms. In this paper, we present a new methodology that combines Haar Like Features - Cascade of Boosted Classifiers, Dense Scale-Invariant Feature Transform (DSIFT), Local Naive Bayes Nearest Neighbor (LNBNN) algorithm for the recognition of human face. We use Haar Like Features and the combination of AdaBoost algorithm and Cascade stratified model to detect and extract the face image, the DSIFT descriptors of the image are computed only for the aligned and cropped face image.Then, we apply the LNBNN algorithms for object recognition. Numerical testing on several benchmark datasets using our proposed method for facerecognition gives the better results than other methods. The accuracies obtained by LNBNN method is 99.74 %.

Abnormality detection of mammograms by discriminative dictionary learning on DSIFT descriptors.

Detection and classification of breast lesions using mammographic images are one of the most difficult studies in medical image processing. A number of learning and non-learning methods have been proposed for detecting and classifying these lesions. However, the accuracy of the detection/classification still needs improvement. In this paper we propose a powerful classification method based on sparse learning to diagnose breast cancer in mammograms. For this purpose, a supervised discriminative dictionary learning approach is applied on dense scale invariant feature transform (DSIFT) features. A linear classifier is also simultaneously learned with the dictionary which can effectively classify the sparse representations. Our experimental results show the superior performance of our method compared to existing approaches.

Facial expression recognition in the wild based on multimodal texture features

Facial expression recognition in the wild is a very challenging task. We describe our work in static and continuous facial expression recognition in the wild. We evaluate the recognition results of gray deep features and color deep features, and explore the fusion of multimodal texture features. For the continuous facial expression recognition, we design two temporal–spatial dense scale-invariant feature transform (SIFT) features and combine multimodal features to recognize expression from image sequences. For the static facial expression recognition based on video frames, we extract dense SIFT and some deep convolutional neural network (CNN) features, including our proposed CNN architecture. We train linear support vector machine and partial least squares classifiers for those kinds of features on the static facial expression in the wild (SFEW) and acted facial expression in the wild (AFEW) dataset, and we propose a fusion network to combine all the extracted features at decision level. The final achievement we gained is 56.32% on the SFEW testing set and 50.67% on the AFEW validation set, which are much better than the baseline recognition rates of 35.96% and 36.08%.

Compressed matching for feature vectors

The problem of compressing a large collection of feature vectors is investigated, so that object identification can be processed on the compressed form of the features. The idea is to perform matching of a query image against an image database, using directly the compressed form of the descriptor vectors, without decompression. Specifically, we concentrate on the Scale Invariant Feature Transform (SIFT), a known object detection method, as well as on Dense SIFT and PHOW features, that contain, for each image, about 300 times as many vectors as the original SIFT. Given two feature vectors, we suggest achieving our goal by compressing them using a lossless encoding by means of a Fibonacci code, for which the pairwise matching can be done directly on the compressed files. In our experiments, this approach improves the processing time and incurs only a small loss in compression efficiency relative to standard compressors requiring a decoding phase.

Dense SIFT for ghost-free multi-exposure fusion

Due to the limited capture range of common imaging sensors, a scene with high dynamic range usually cannot be well described by a single still image because some regions in it may be under-exposed or over-exposed. In this paper, a new multi-exposure fusion method based on dense scale invariant feature transform (SIFT) is presented. In our algorithm, the dense SIFT descriptor is first employed as the activity level measurement to extract local details from source images, and then adopted to remove ghosting artifacts when the captured scene is dynamic with moving objects. Furthermore, two popular weight distribution strategies for local contrast extraction, namely, “weighted-average” and “winner-take-all” are studied in this paper. The effects of these two strategies on the fusion results are compared and discussed. Experimental results demonstrate the effectiveness of the proposed method in terms of both visual quality and objective evaluation.

Multi-focus image fusion with dense SIFT

Multi-focus image fusion technique is an important approach to obtain a composite image with all objects in focus. The key point of multi-focus image fusion is to develop an effective activity level measurement to evaluate the clarity of source images. This paper proposes a novel image fusion method for multi-focus images with dense scale invariant feature transform (SIFT). The main novelty of this work is that it shows the great potential of image local features such as the dense SIFT used for image fusion. Particularly, the local feature descriptor can not only be employed as the activity level measurement, but also be used to match the mis-registered pixels between multiple source images to improve the quality of the fused image. In our algorithm, via the sliding window technique, the dense SIFT descriptor is first used to measure the activity level of source image patches to obtain an initial decision map, and then the decision map is refined with feature matching and local focus measure comparison. Experimental results demonstrate that the proposed method can be competitive with or even outperform the state-of-the-art fusion methods in terms of both subjective visual perception and objective evaluation metrics.

A new feature extraction framework based on wavelets for breast cancer diagnosis

This paper investigates a pattern recognition framework in order to determine and classify breast cancer cases. Initially, a two-class separation study classifying normal and abnormal (cancerous) breast tissues is achieved. The Histogram of Oriented Gradients (HOG), Dense Scale Invariant Feature Transform (DSIFT), and Local Configuration Pattern (LCP) methods are used to extract the rotation- and scale-invariant features for all tissue patches. A classification is made utilizing Support Vector Machine (SVM), k-Nearest Neighborhood (k-NN), Decision Tree, and Fisher Linear Discriminant Analysis (FLDA) via 10-fold cross validation. Then, a three-class study (normal, benign, and malignant cancerous cases) is carried out using similar procedures in a two-class case; however, the attained classification accuracies are not sufficiently satisfied. Therefore, a new feature extraction framework is proposed. The feature vectors are again extracted with this new framework, and more satisfactory results are obtained. Our new framework achieved a remarkable increase in recognition performance for the three-class study.

An Image Classification Algorithm Based on Bag of Visual Words and Multi-kernel Learning

In this article, we propose an image classification algorithm based on Bag of Visual Words model and multi-kernel learning. First of all, we extract the D-SIFT (Dense Scale-invariant Feature Transform) features from images in the training set. And then construct visual vocabulary via K-means clustering. The local features of original images are mapped to vectors of fixed length through visual vocabulary and spatial pyramid model. At last, the final classification results are given by generalized multiple kernel proposed by this paper. The experiments are performed on Caltech-101 image dataset and the results show the accuracy and effectiveness of the algorithm

Automated mitosis detection using texture, SIFT features and HMAX biologically inspired approach

Context: According to Nottingham grading system, mitosis count in breast cancer histopathology is one of three components required for cancer grading and prognosis. Manual counting of mitosis is tedious and subject to considerable inter- and intra-reader variations. Aims: The aim is to investigate the various texture features and Hierarchical Model and X (HMAX) biologically inspired approach for mitosis detection using machine-learning techniques. Materials and Methods: We propose an approach that assists pathologists in automated mitosis detection and counting. The proposed method, which is based on the most favorable texture features combination, examines the separability between different channels of color space. Blue-ratio channel provides more discriminative information for mitosis detection in histopathological images. Co-occurrence features, run-length features, and Scale-invariant feature transform (SIFT) features were extracted and used in the classification of mitosis. Finally, a classification is performed to put the candidate patch either in the mitosis class or in the non-mitosis class. Three different classifiers have been evaluated: Decision tree, linear kernel Support Vector Machine (SVM), and non-linear kernel SVM. We also evaluate the performance of the proposed framework using the modified biologically inspired model of HMAX and compare the results with other feature extraction methods such as dense SIFT. Results: The proposed method has been tested on Mitosis detection in breast cancer histological images (MITOS) dataset provided for an International Conference on Pattern Recognition (ICPR) 2012 contest. The proposed framework achieved 76% recall, 75% precision and 76% F-measure. Conclusions: Different frameworks for classification have been evaluated for mitosis detection. In future work, instead of regions, we intend to compute features on the results of mitosis contour segmentation and use them to improve detection and classification rate.

Scale Invariant Feature Transform

Scale Invariant Feature Transform (SIFT) is an image descriptor for image-based matching developed by David Lowe (1999,2004). This descriptor as well as related image descriptors are used for a large number of purposes in computer vision related to point matching between different views of a 3-D scene and view-based object recognition. The SIFT descriptor is invariant to translations, rotations and scaling transformations in the image domain and robust to moderate perspective transformations and illumination variations. Experimentally, the SIFT descriptor has been proven to be very useful in practice for robust image matching and object recognition under real-world conditions.In its original formulation, the SIFT descriptor comprised a method for detecting interest points from a grey-level image at which statistics of local gradient directions of image intensities were accumulated to give a summarizing description of the local image structures in a local neighbourhood around each interest point, with the intention that this descriptor should be used for matching corresponding interest points between different images. Later, the SIFT descriptor has also been applied at dense grids (dense SIFT) which have been shown to lead to better performance for tasks such as object categorization and texture classification. The SIFT descriptor has also been extended from grey-level to colour images and from 2-D spatial images to 2+1-D spatio-temporal video.