<h3>Abstract</h3> The prediction of acid dissociation constants (p<i>K</i><sub>a</sub>) is a prerequisite for predicting many other properties of a small molecule, such as its protein-ligand binding affinity, distribution coefficient (log <i>D</i>), membrane permeability, and solubility. The prediction of each of these properties requires knowledge of the relevant protonation states and solution free energy penalties of each state. The SAMPL6 p<i>K</i><sub>a</sub> Challenge was the first time that a separate challenge was conducted for evaluating p<i>K</i><sub>a</sub> predictions as part of the Statistical Assessment of Modeling of Proteins and Ligands (SAMPL) exercises. This challenge was motivated by significant inaccuracies observed in prior physical property prediction challenges, such as the SAMPL5 log <i>D</i> Challenge, caused by protonation state and p<i>K</i><sub>a</sub> prediction issues. The goal of the p<i>K</i><sub>a</sub> challenge was to assess the performance of contemporary p<i>K</i><sub>a</sub> prediction methods for drug-like molecules. The challenge set was composed of 24 small molecules that resembled fragments of kinase inhibitors, a number of which were multiprotic. Eleven research groups contributed blind predictions for a total of 37 p<i>K</i><sub>a</sub> distinct prediction methods. In addition to blinded submissions, four widely used p<i>K</i><sub>a</sub> prediction methods were included in the analysis as reference methods. Collecting both microscopic and macroscopic p<i>K</i><sub>a</sub> predictions allowed in-depth evaluation of p<i>K</i><sub>a</sub> prediction performance. This article highlights deficiencies of typical p<i>K</i><sub>a</sub> prediction evaluation approaches when the distinction between microscopic and macroscopic p<i>K</i><sub>a</sub>s is ignored; in particular, we suggest more stringent evaluation criteria for microscopic and macroscopic p<i>K</i><sub>a</sub> predictions guided by the available experimental data. Top-performing submissions for macroscopic p<i>K</i><sub>a</sub> predictions achieved RMSE of 0.7-1.0 p<i>K</i><sub>a</sub> units and included both quantum chemical and empirical approaches, where the total number of extra or missing macroscopic p<i>K</i><sub>a</sub>s predicted by these submissions were fewer than 8 for 24 molecules. A large number of submissions had RMSE spanning 1-3 p<i>K</i><sub>a</sub> units. Molecules with sulfur-containing heterocycles or iodo and bromo groups were less accurately predicted on average considering all methods evaluated. For a subset of molecules, we utilized experimentally-determined microstates based on NMR to evaluate the dominant tautomer predictions for each macroscopic state. Prediction of dominant tautomers was a major source of error for microscopic p<i>K</i><sub>a</sub> predictions, especially errors in charged tautomers. The degree of inaccuracy in p<i>K</i><sub>a</sub> predictions observed in this challenge is detrimental to the protein-ligand binding affinity predictions due to errors in dominant protonation state predictions and the calculation of free energy corrections for multiple protonation states. Underestimation of ligand p<i>K</i><sub>a</sub> by 1 unit can lead to errors in binding free energy errors up to 1.2 kcal/mol. The SAMPL6 p<i>K</i><sub>a</sub> Challenge demonstrated the need for improving p<i>K</i><sub>a</sub> prediction methods for drug-like molecules, especially for challenging moieties and multiprotic molecules.