Spectral-Spatial Feature Extraction for Hyperspectral Image Matching and Boundary Detection

Abstract

Listen

Translate article

A hyperspectral image contains a huge amount of information compared to grayscale and RGB images thanks to its high spectral resolution and wide sensing spectrum. This facilitates analysis and interpretation of properties and features of specific materials in the image. Exploiting both spectral and spatial information can provide more comprehensive and discriminative characteristics of objects of interest than traditional methods. Recently, hyperspectral imaging has been used in many applications such as medicine, agriculture, environment and astronomy. Furthermore, due to the availability and cost reduction of imaging devices, hyperspectral imaging has also been introduced into computer vision and can be adopted in many pattern recognition tasks, for example, object detection, recognition and segmentation. Therefore, the demand for developing new methods that explore all information of hyperspectral images has increased. Accordingly, the main target of this thesis is to extract effective and robust spectral-spatial features from close-range hyperspectral images to facilitating some of the pattern recognition and image processing tasks. In this thesis, we propose three novel spectral-spatial feature extraction methods for image matching and boundary detection. First, we exploit both spectral and spatial dimensions simultaneously by extracting 3D features from close-range hyperspectral images. These features are spectral and geometric transformation invariants that can be used to register hyperspectral images with different spectral conditions and different geometric projections. This method is named Spectral-Spatial Scale Invariant Feature Transform (SS-SIFT). Similar to the classic SIFT algorithm, SS-SIFf consists of keypoint detection and descriptor construction steps. Our main contribution to this method is extracting 3D keypoints from spectral-spatial scale space by detecting them from extrema after 3D difference of Gaussian is applied to the data cube. Furthermore, for each keypoint, two descriptors are proposed by exploring the distribution of spectral-spatial gradient magnitudes in its local 3D neighbourhood. SS-SIFT features are used effectively to match hyperspectral images that are captured with different light conditions, viewing angles, and using two different hyperspectral cameras. The second work of this thesis is extracting spectral-spatial features for boundary detection in close-range hyperspectral images. Boundary detection is a fundamental task in computer vision and numerous research has been proposed in RGB images. However, there is a dearth of research on boundary detection in hyperspectral image due to high data dimensionality and the complexity of information that is distributed over the spectral bands. Thus, we propose a spectral-spatial feature based statistical co-occurrence method for this task. We extract simple and significant features from both spectral and spatial dimensions at the same time to describe both structure and material properties. Each pixel vector is converted to a feature space based on its surrounding neighbours. Then, we adopt the probability density function to estimate the co-occurrence of features at neighbouring pixel pairs. Then spectral-spatial affinity matrix is constructed based on probability density function values. After that, a spectral clustering algorithm is applied on the affinity matrix to solve the eigenproblem and calculate the eigenvectors corresponding to the smallest eigenvalues. Finally, the boundary map is constructed from the most informative eigenvectors. Our proposed algorithm is very effective in exploring object boundaries from close-range hyperspectral images, considering both object material and structure. Lastly, we propose a novel method for effective and efficient boundary detection in close-range hyperspectral images. Considering that the reflectance of materials and their abundances provide significant details of the data, our algorithm investigates the potential of integrating hyperspectral unmixing results with the spectral responses to provide effective spectral-spatial feature. We use a nonnegative matrix factorization method to estimate the material abundances and then fuse them linearly with material reflectance to provide spectral-spatial features. We subsequently construct a spectral-spatial affinity matrix based on calculating the efficient spectral similarity measurements between the feature vectors of neighbouring pixels within a local neighbourhood. After that, the eigenproblem of the affinity matrix is solved and eigenvectors of the smallest eigenvalues are calculated. Finally, we construct the boundary map from the most informative eigenvectors. Regardless of the efficiency of the model, both proposed boundary detection methods are effective to predict boundaries of objects with similar colour but different materials and can cope with several scenarios which methods based on colour images cannot handle.