A key challenge in epidemic modeling is the lack of adequate data on population interactions (such as traffic flow or mobility patterns) which result in the spread of infectious diseases. Knowledge of social contact patterns is crucial for public health professionals to devise effective non-pharmaceutical interventions to control epidemics. This paper focuses on inferring social contact rates from reported infection counts during the spread of an infectious disease, addressing the increased difficulty that arises when dealing with “sparse” contact networks where only a small subset of the edges have non-zero weights. Specifically, a new geographically constrained lasso approach for network reconstruction for non-homogeneous mixing Susceptible-Infected-Removed (SIR) disease spread models is presented. The new network reconstruction method can explicitly account for the spatial proximity of network nodes in estimating the disease transmission rates and predicting the future evolution of the epidemic dynamics. Extensive numerical experiments are presented to show the proposed method outperforms existing approaches in terms of accuracy of contact identification under various graph topologies. A case study based on real data from the COVID-19 pandemic is presented to demonstrate the application of the approach for inferring contact structures and a counterfactual scenario analysis to assess effectiveness of containment strategies.