Mitochondria are recognized as an important target organelle for the toxicity of nanomaterials. Although the toxic effects of silver nanoparticles (AgNPs) on mitochondria have been widely reported, the mechanism behind the toxicity remains unclear. In this study, the effects of two forms of silver (AgNPs and AgNO3) on carp gill mitochondria were investigated by analyzing the mitochondrial ultrastructure, physicochemical properties of mitochondrial membrane, and mitochondrial proteomics. After exposure of common carp to AgNPs (0.75 mg/L) and AgNO3 (0.05 mg/L) for 96 h, both forms of silver were shown to cause gill mitochondrial lesions, including irregular shape, loss of mitochondrial cristae, and increased mitochondrial membrane permeability. Proteomics results revealed that AgNPs and AgNO3 induced 362 and 297 differentially expressed proteins (DEPs) in gill mitochondria, respectively. Among the DEPs, 244 were shared between AgNPs and AgNO3 treatments. These shared proteins were mainly distributed in the mitochondrial membrane and matrix, and were significantly enriched in the tricarboxylic acid (TCA) cycle and oxidative phosphorylation pathway. The functional annotation of DEPs induced by both silver forms was mainly involved in energy production and conversion. These results indicated that the toxic mechanism of AgNPs and AgNO3 on gill mitochondria were comparable and the two forms of silver caused mitochondrial dysfunction in fish gills by inhibiting the TCA cycle and disrupting the electron transport chain.