Is matching in different situations equally applicable for impact evaluation studies when using observational data

Abstract

Under certain circumstances, randomisation of interventions may not be applicable in development activities. Quasi-experimental research design such as matching often helps creating counterfactuals for impact evaluations. Although different types of statistical matching methods are available, their relative performance is generally unknown to practitioners. Using five sets of household survey data collected from samples of treatment and comparison groups from four countries in the Hindu Kush Himalaya region, we examine the extent of covariate imbalances before and after matching using four different matching methods. For small samples with enough imbalances in the covariates, the nearest neighbour matching does not perform well, but matching with stratification works better. The performance of radius and kernel matching falls in between nearest neighbour and matching with stratification. We find that the matching is useful when covariates imbalance is high before matching but less useful for sample with relatively balanced covariates before matching.