On January 16, 2021, India rolled out the COVID vaccination drive. A successful and effective vaccination campaign requires much more than the availability of a safe and effective vaccine. This includes identifying vulnerable populations with lower vaccine confidence and identifying the drivers of vaccine hesitancy. This study aims to find out vaccine hesitancy among the tribal population regarding COVID-19 vaccination. It was an observational descriptive cross-sectional study, conducted at Manindranagar and Hatinagar gram panchayat of Berhampore Block of Murshidabad district, West Bengal, from June 2021-November 2021, among tribal people aged >18 years. A total of 198 tribal people were selected by applying the probability proportional to size sampling method. Participants were interviewed using predesigned, pretested, and semi-structured schedules. Potential predictors of hesitancy were investigated using the multivariate logistic regression model. Vaccine hesitancy was present among 36.9% of the study participants. Fear of side effects (78.1%) was the most common reason of vaccine hesitancy. Only 30.8% of them received at least one dose of vaccine. Vaccine hesitancy was associated with decreased family income in the last 1 year (adjusted odds ratio [AOR] = 8.23), knowledge regarding vaccine (AOR = 0.41), adherence to COVID-appropriate behavior (AOR = 0.45), and trust on the local health-care worker (AOR = 0.32). Vaccine hesitancy among the tribal population is driven by a lack of knowledge and awareness. Their economic status, attitudes toward the health system, and accessibility factors may also play a major role in vaccine hesitancy. Extensive information, education, and communication activity, more involvement of health-care workers in the awareness campaign, and establishment of vaccination centers in tribal villages may be helpful.