The Harpy connected speech recognition system is the result of our attempt to understand the relative importance of various design choices of two earlier speech recognition systems developed at Carnegie-Mellon University, The Hearsay-I system (D. R. Reddy, et al., IEEE Trans. AU, 229–238 (1973); L. D. Erman, Technical Report, Computer Science Department, Carnegie-Mellon University, (1974) and the Dragon system [J. K. Baker, Ph.D. Thesis (Carnegie-Mellon University)]. Knowledge is represented as procedures in Hearsay-I and as a Markov network with a-priori transition probabilities between states in Dragon. Hearsay-I uses a best first search while Dragon searches all the possible (acoustic syntactic) paths through the network to determine the optimal path. Hearsay-I uses segmentation and labeling and Dragon is a segmentation-free system. Systematic performance analysis of various design choices resulted in the Harpy system which represents knowledge as a finite state transition network but without the a-priori transition probabilities, searches only a few “best” paths, and uses segmentation to reduce the number of state probability updates that must be done. The system achieves between 70% and 100% sentence accuracy, depending upon the task, and runs between 1.5 and 10 times real time. Complete details of design, implementation, and experimental results are given in B. P. Lowerre, Ph.D. Thesis (Computer Science Department, Carnegie-Mellon University, 1976).