We consider the problem of building a transitional model of an initially uncalibrated camera network. More specifically, we discuss a Hidden Markov Model (HMM) based strategy in which the model's state-space is defined in terms of a partition of the physical network coverage. Transitions between any two such states are described by the distribution of the underlying Markov Process. Extending previous work in (Cenedese et al., 2010), we show how it is possible to infer the model structure and parameters from coordinate free observations and we introduce a novel performance index for model validation. We moreover show the predictive power of this HMM approach in simulated and real settings that comprise Pan-Tilt-Zoom (PTZ) cameras.