Semisupervised learning (SSL) methods have shown great potential in the hyperspectral classification with a limited number of labeled samples. In this paper, we suggest a new graph-based SSL (GBSSL) using both spectral and spatial information. In the first step, we constructed two graphs using spectral and spatial information. Then, the Laplacians of both spectral and spatial graphs were merged to form a weighted joint graph. To improve the quality of spatial neighborhood for conforming the image objects, we employed the adaptive neighborhood (AN) technique. Instead of using the conventional crisp spatial neighborhood, the flat zone area filtering approach was used to define AN and extract the spatial information. By this way, each pixel is only connected to the pixels of a single flat zone, which presents a particular object in the image. Consequently, the border between different classes is extracted more precisely. As a result, the final classified map is more homogenous, and the salt and pepper effect is removed. To evaluate the efficiency of the proposed method, the experiments were carried out on three real benchmark hyperspectral data sets with different types of land cover, and different spectral and spatial resolutions. The results of the proposed method showed excellent performances in all data sets, specifically where a very limited number of labeled training samples were available. This method achieves a significant improvement compared to the state-of-the-art classifiers such as SVM, spectral–spatial SVM, and spectral–spatial GBSSL.