The newly developed hyperspherical color space (HCS) fusion method is a cost-effective panchromatic (pan) sharpening technique, but it still suffers from local spectral distortion and insufficient overall spatial details. This paper attempts to improve the HCS fusion method by combining the spectrum gain modulation and wavelet transformation and presents a novel fusion method called enhanced HCS (EHCS) to mitigate the mentioned issue. This method mainly contains three steps. (1) The proposed method extracts the angular component from multispectral images through hyperspherical color transformation (HCT). (2) A modulated multispectral image is generated by using the spectrum gain modulation. Then, the spatial details of the modulated multispectral image are substituted with the spatial details of the pan image by multiscale wavelet decomposition. (3) Finally, the angular component and modulated detail image are integrated to achieve the fused image by the inverse HCT. To validate the proposed method, three high spatial resolution images, i.e., two WorldView-2 and one QuickBird datasets, acquired at Fujian, China, were used. The comparison with HCS method was also done. Both the visual and quantitative analysis demonstrate that the EHCS method can significantly preserve the spectral characteristics and enhance the spatial detail as well.