Multi-Target Tracking with Probabilistic Graphical Models

Abstract

Thanks to revolutionary developments in microscopy techniques such as robotic high-throughput setups or light sheet microscopy, vast amounts of data can be acquired at unprecedented temporal and spatial resolution. The mass of data naturally prohibits manual analysis, though, and life scientists thus have to rely more and more on automated cell tracking methods. However, automated cell tracking involves intricacies that are not commonly found in traditional tracking applications. For instance, cells may undergo mitosis, which results in variable numbers of tracking targets across successive frames. These difficulties have been addressed by tracking-by-assignment models in the past, which dissect the task into two stages, detection and tracking. However, as with every two-stage framework, the approach hinges on the quality of the first stage, and errors propagate partially irrevocably from the detection to the tracking phase. The research in this thesis thus focuses on methods to advance tracking-by-assignment models in order to avoid these errors by exploiting synergy effects between the two (previously) separate stages. We propose two approaches, both in terms of probabilistic graphical models, which allow for information exchange between the detection and the tracking step to different degrees. The first algorithm, termed Conservation tracking, models both possible over- and undersegmentation errors and implements global consistency constraints in order to reidentify target identities even across occlusion or erroneous detections. Wrong detections from the first step can hence be corrected in the second stage. The second method goes one step further and optimizes the two stages completely jointly in one holistic model. In this way, the detection and tracking step can maximally benefit from each other and reach the overall most likely interpretation of the data. Both algorithms yield notable results which are state-of-the-art. In spite of the distinguished results achieved with these methods, automated cell tracking methods are still error-prone and manual proof-reading is often unavoidable for life scientists. To avoid the time-consuming manual identification of errors on very large datasets, most ambiguous predictions ought to be detected automatically so that these can be corrected by a human expert with minimal effort. In response, we propose two easy-to-use methods to sample multiple solutions from a tracking-by-assignment graphical model and derive uncertainty measures from the variations across the samples. We showcase the usefulness for guided proof-reading on the cell tracking model proposed in this work. Finally, the successful application of structured output learning algorithms to cell tracking in previous work inspired us to advance the state-of-the-art by an algorithm called Coulomb Structured Support Vector Machine (CSSVM). The CSSVM improves the expected generalization error for unseen test data by the training of multiple concurrent graphical models. Through the novel diversity encouraging term, motivated from experimental design, the ensemble of graphical models is learned to yield diverse predictions for test data. The best prediction amongst these models may then be selected by an oracle or with respect to a more complex loss. Experimental evaluation shows significantly better results than using only one model and achieves state-of-the-art performance on challenging computer vision tasks.