Modeling the endpoint of binary black hole mergers is a cornerstone of modern gravitational-wave astronomy. Extracting multiple quasinormal mode frequencies from the ringdown signal allows the remnant black hole to be studied in unprecedented detail. Previous studies on numerical relativity simulations of aligned-spin binaries have found that it is possible to start the ringdown analysis much earlier than previously thought if overtones (and possibly mirror modes) are included. This increases the signal-to-noise ratio in the ringdown making identification of subdominant modes easier. In this paper we study, for the first time, black hole binaries with misaligned spins and find a much greater variation in the performance of ringdown fits than in the aligned-spin case. The inclusion of mirror modes and higher harmonics, along with overtones, improves the reliability of ringdown fits with an early start time; however, there remain cases with poor performing fits. While using overtones in conjunction with an early ringdown start time is an enticing possibility, it is necessary to proceed with caution. We also consider for the first time the use of numerical relativity surrogate models in this type of quasinormal mode study and address important questions of accuracy in the underlying numerical waveforms used for the fit.