Measurement properties of key constructs used in tobacco retailer audit studies are infrequently examined. We used diagnostic testing metrics to assess the predictive validity of product advertising and availability. In 2017, we documented tobacco product advertising and availability in a sample of New York City and New Jersey retailers (N = 987). We calculated positive predictive values (PPV, ie, probability of availability given advertising is present) and sensitivity scores (ie, probability of advertising given product is available) for cigarettes, cigars, smokeless tobacco, and electronic nicotine delivery systems (ENDS). Advertising presence was an imprecise indicator of availability for ENDS (PPV: 82.3%) and smokeless tobacco (PPV: 88.2%), particularly in non-chain convenience stores. Cigars were least likely to be advertised, if available (sensitivity: 27.8%), compared to other product types (sensitivity range: 43.8% - 57.9%). The suboptimal predictive validity of advertising for certain products suggests that this measure does not necessarily reflect product accessibility in communities. Moreover, differences in sensitivity scores suggest that some product classes rely on strategies other than physical ads to recruit and retain consumers. A better understanding of these constructs and their influence on behavior can help inform more effective retail-based regulatory approaches.