A popular phenomenon in the street-hailing taxi system is the imbalanced mobility services between city central and outside downtown areas, which leads to unmet demand outside downtown areas and competitions in city central areas. Understanding taxi drivers’ customer-searching behaviors is crucial to addressing the phenomenon and redistributing the taxi supply. However, the current literature ignores or simply models the taxi drivers’ behaviors, in particular, lacks the in-depth discussions on individuals’ heterogeneity. This study introduces the latent class model to identify the internal and external factors influencing the taxi drivers’ destination choice after the last drop-offs. Beyond the influencing factors, the modeling structure captures the heterogeneity in vacant taxicab drivers through introducing latent classes. The proposed model outperforms other discrete choice models, for instance, multinomial logit, nested logit, and mixed logit, based on the two study cases developed from the New York City yellow taxicab system. The empirical results first statistically indicate the existence of latent classes, which further empirically prove the heterogeneity in the choices by vacant taxicab drivers while searching customers. Moreover, we obtain a set of internal and external factors influencing the customer searching behaviors. For example, the taxicab drivers are sensitive to the demand at the search destination areas and the distance from the last drop-off location to the search destination areas and behave identically in particular under the conditions of high demand and short search distance. On the other hand, the external variables have different impacts on customer searching behaviors across the different groups of drivers in the both study cases, including peak hours, weekday, holiday, earned fare from last occupied trip, raining hours, and flight arrivals at airports. In final, the proposed modeling structure and findings are useful as a sub-model of taxi system modeling while developing strategies, as well as as a regional planning tool for taxi supply estimations.