Detecting near-Earth asteroids (NEAs) is crucial for research in solar system and planetary science. In recent year, deep-learning methods have almost dominated the task. Since NEAs represent only one-thousandth of the pixels in images, we proposed an ICC-BiFormer model that includes an image compression and contrast enhancement block and a BiFormer model to capture local features in input images, which is different from previous models based on Convolutional Neural Network (CNN). Furthermore, we utilize a larger input size of the model, which corresponds to the side length of the input image matrix, and design a cropping algorithm to prevent NEAs from being truncated and better divide NEAs and satellites. We apply our ICC-BiFormer model into a dataset of approximately 20,000 streak and 40,000 non-streak images to train a binary classification model. The ICC-BiFormer achieves 99.88% accuracy, which is superior to existing models. Focusing on local features has been proven effective in detecting NEAs.