Many horticultural tasks require a definition of and means to measure ‘plant state’, a quantity that is used to represent and compare plants. The plant state's evolution through time (plant growth) can then be tracked. This is often performed informally by humans based on heuristic features for particular plant types. It can be used for example to recommend interventions to catch up on expected growth, or to measure the effects of experimental interventions on growth. This work provides a purely data-driven definition, that is easy to train, easy to non-destructively acquire training data, does not require expert annotations, and is easy to compute for any new plant type and growing conditions. The presented method is applied to a dataset of lettuce plants where it exceeds the performance of a hard baseline. This work also demonstrates that the presented method retains the intuitive properties expected for a plant growth model. Open source code implementation and data is provided.