Sparse unmixing (SU) algorithms use the existing spectral library as prior knowledge to analyze the endmembers and estimate abundance maps. The majority of SU algorithms utilize loss functions based on <i>L</i><sub>2,1</sub>-norm or <i>F</i>-norm to minimize reconstruction error. They have different advantages and shortcomings. In short, <i>F</i>-norm has a differentiable characteristic, and it is easy to minimize as a loss function. However, it is very sensitive to heavy noise and outliers. While the <i>L</i><sub>2,1</sub>-norm emphasizes the reconstruction error on each band and is robust to noise with different intensities in different bands. But the <i>L</i><sub>2,1</sub>-norm is non-differentiable at zero-point. This paper introduces an adaptive loss function based on σ-norm for SU, which combines the advantages of <i>L</i><sub>2,1</sub>-norm and <i>F</i>-norm. The adaptive loss function is related to a non-negative parameter σ. By adjusting the parameter σ, the adaptive loss function can approach <i>F</i>-norm or <i>L</i><sub>2,1</sub>-norm. To the best of our knowledge, it is the first time to apply an adaptive loss function to SU. Moreover, the adaptive loss function is globally differentiable, and we propose an optimization algorithm for the adaptive loss function and verify its convergence. Experiments on the real-world and synthetic HSIs show that the adaptive loss function effectively enhances the performance of the sparse unmixing algorithms.