Indoor air quality (IAQ) has a significant influence on occupants' comfort, health, productivity, and safety. Existing studies show that the primary causes of many IAQ problems are various airborne contaminants that either are generated indoors or penetrate into indoor environments with passive or active airflows. Accurate and prompt identification of contaminant sources can help determinate appropriate IAQ control solutions, such as, eliminating contaminant sources, isolating and cleaning contaminated spaces. This study develops a fast and effective inverse modeling method for identifying indoor contaminant source characteristics. The paper describes the principles of the probability-based adjoint inverse modeling method and formulates a multi-zone model based inverse prediction algorithm that can rapidly track contaminant source location with known source release time in a building with many compartments. The paper details the inverse modeling procedure with modification of an existing multi-zone airflow and contaminant transport simulation program. The application of the method has been demonstrated with two case studies: contaminant releases in a multi-compartment residential house and in a complex institutional building. The numerical experiments tested the source identification capability of the program for various contaminant sensing scenarios. The investigation verifies the effectiveness and accuracy of the developed method for indoor contaminant source tracking, which will be further explored to identify more complicated indoor contamination episodes.