An increased reliance on imagery as the source of biodiversity data from the deep sea has stimulated many recent advances in image annotation and data management. The form of image-derived data is determined by the way faunal units are classified and should align with the needs of the ecological study to which it is applied. Some applications may require only low-resolution biodiversity data, which is easier and cheaper to generate, whereas others will require well-resolved biodiversity measures, which require a larger investment in annotation methods. We assessed these trade-offs using a dataset of 5 939 images and physical collections of black and octocorals taken during surveys from a seamount area in the southwest Pacific Ocean. Coral diversity was greatly underestimated in images: only 55 black and octocoral ‘phototaxa’ (best-possible identifications) were consistently distinguishable out of a known 210 species in the region (26%). Patterns of assemblage composition were compared between the phototaxa and a standardized Australian classification scheme (“CATAMI”) that uses morphotypes to classify taxa. Results were similar in many respects, but the identities of dominant, and detection of rare but locally abundant, coral entities were achieved only when annotation was at phototaxon resolution, and when faunal densities were recorded. A case study of data from 4 seamounts compared three additional classification schemes. Only the two with highest resolution – phototaxon and a combined phototaxon-morphological scheme – were able to distinguish black and octocoral communities on unimpacted vs. impacted seamounts. We conclude that image annotation schemes need to be fit-for-purpose. Morphological schemes such as CATAMI may perform well and are most easily standardized for cross-study data sharing, but high resolution (and more costly) annotation schemes are likely necessary for some ecological and management-based applications including biodiversity inventory, change detection (monitoring) – and to develop automated annotation using machine learning.