Sesame (Sesamum indicum) is an important oilseed crop with rising demand owing to its nutritional and health benefits. There is an urgent need to develop and integrate new genomic-based breeding strategies to meet these future demands. While genomic resources have advanced genetic research in sesame, the implementation of high-throughput phenotyping and genetic analysis of longitudinal traits remains limited. Here, we combined high-throughput phenotyping and random regression models to investigate the dynamics of plant height, leaf area index, and five spectral vegetation indices throughout the sesame growing seasons in a diversity panel. Modeling the temporal phenotypic and additive genetic trajectories revealed distinct patterns corresponding to the sesame growth cycle. We also conducted longitudinal genomic prediction and association mapping of plant height using various models and cross-validation schemes. Moderate prediction accuracy was obtained when predicting new genotypes at each time point, and moderate to high values were obtained when forecasting future phenotypes. Association mapping revealed three genomic regions in linkage groups 6, 8, and 11, conferring trait variation over time and growth rate. Furthermore, we leveraged correlations between the temporal trait and seed-yield and applied multi-trait genomic prediction. We obtained an improvement over single-trait analysis, especially when phenotypes from earlier time points were used, highlighting the potential of using a high-throughput phenotyping platform as a selection tool. Our results shed light on the genetic control of longitudinal traits in sesame and underscore the potential of high-throughput phenotyping to detect a wide range of traits and genotypes that can inform sesame breeding efforts to enhance yield.