Glacier surface velocity is an important variable for glacier dynamics studies. Estimation of accurate surface velocity from remote sensing is a challenge, especially for glaciers with no in-situ observations. To overcome this challenge, a new method for glacier feature tracking named as SWIFT (Spatially varying WIndow based maximum likelihood Feature Tracking) has been proposed. This method utilises both optical data (to automatically determine the window size using the concept of Object Based Image Analysis) and Synthetic Aperture Radar (SAR) data (to perform feature tracking). The proposed method uses a spatially varying window size unlike other existing softwares that cannot provide the flexibility of a spatially varying window size. The proposed method has been tested and validated at three different glaciers (South Glacier, Canada; Chhota Shigri Glacier, India; and Tasman Glacier, New Zealand) for which field measured data were available. The obtained results for all three glaciers showed consistent improvement in estimated velocity by SWIFT when compared with spatially fixed window size-based estimates from normalized cross correlation-based Correlation Image Analysis Software (CIAS). Considering the data availability, the proposed SWIFT method has been implemented using a variety of SAR and optical satellite data to understand its performance/effectiveness for glacier surface velocity estimation. When validated against field measurements, the results from SWIFT gave an RMSE of 12.8 m/yr, 15.32 m/yr and 67.1 m/yr for South Glacier, Chhota Shigri Glacier and Tasman Glacier, respectively. Moreover, the RMSE of SWIFT estimates were observed to have an RMSE that was 19%-36% lower than the best performing spatially fixed window size.