A general problem when modifying knowledge bases is that changes may degrade system performance. This is especially a problem when the knowledge base is large; it may be unclear how changing one item in a knowledge base containing thousands of items will affect overall system performance. Aquinas, a knowledge acquisition tool, uses knowledge elicitation and representation techniques and consultation review mechanisms to help alleviate this problem. The consultation review mechanisms are discussed here. We are experimenting with ways to use consultations and test cases to refine the information in an Aquinas knowledge base. The domain expert can use interactive graphics to specify the expected results. Modifications to the knowledge base may be tested against previous consultations; adjustments are suggested that make the results of all previous consultations as well as the current consultation correlate better with the expert's expectations. New traits are synthesized that would improve the performance of all previous consultations. New test cases are suggested that cover aspects missed by previous test cases. While we are just beginning to experiment with these techniques, they promise to provide help in improving problem-solving performance and gaining problem-solving insight.