The advent of high throughput sequencing has permitted to investigate the genome and the transcriptome of novel non-model species with unprecedented depth. This technological advance provided a better understanding of the evolution of adaptive immune genes in gnathostomes, revealing several unexpected features in different fish species which are of particular interest. In the present paper, we review the current understanding of the adaptive immune system of the coelacanth, the elephant shark and the Atlantic cod. The study of coelacanth, the only living extant of the long thought to be extinct Sarcopterygian lineage, is fundamental to bring new insights on the evolution of the immune system in higher vertebrates. Surprisingly, coelacanths are the only known jawed vertebrates to lack IgM, whereas two IgD/W loci are present. Cartilaginous fish are of great interest due to their basal position in the vertebrate tree of life; the genome of the elephant shark revealed the lack of several important immune genes related to T cell functions, which suggest the existence of a primordial set of TH1-like cells. Finally, the Atlantic cod lacks a functional major histocompatibility II complex, but balances this evolutionary loss with the expansion of specific gene families, including MHC I, Toll-like receptors and antimicrobial peptides. Overall, these data point out that several fish species present an unconventional adaptive immune system, but the loss of important immune genes is balanced by adaptive evolutionary strategies which still guarantee the establishment of an efficient immune response against the pathogens they have to fight during their life.