Few-mode fibers (FMF) has unique advantages in single fiber imaging (SFI) in special applications. However, the limited number of transmission modes restricts the information carriers, hence the high-quality image restoration is a major challenge. A deep learning network with new comprehensive attention, called few-mode fiber Swin Transformer (FFST), is developed based on Swin Transformer V2 to realize successful image restoration with a FMF on handwriting letters and numbers. This network is proved outperforming nine widely used convolutional neural networks or Transformer-based models with four evaluation criteria. In addition, single-fiber-multi-location (SFML) experiment is conducted to investigate the effect of different view on imaging feasibility, the result of which shows that SFML can improve the restoration quality despite one object corresponds to multiple different speckle images, indicating that multi-mapping relationships has constructive effect in single FMF imaging. This paper provides a method for imaging in very narrow cavities.