Lampreys are jawless fishes that diverged ∼550 million years ago from other vertebrates. Sequencing of the somatic and the germline genomes of the sea lamprey (Petromyzon marinus) in 2013 and 2018, respectively, has helped to improve our understanding of the genes and gene networks that control many aspects of lamprey development. However, little is known about the genetic basis of gonadal differentiation in lampreys, partly due to the prolonged period during which their gonads remain sexually indeterminate. We performed RNA-sequencing on gonadal samples from four chestnut lamprey (Ichthyomyzon castaneus) and six northern brook lamprey (I. fossor) to identify differentially expressed genes (DEG’s) and pathways associated with transcriptomic differences in: (1) larvae during early gonadal differentiation versus definitive females (i.e., with oocytes in the slow cytoplasmic growth phase); and (2) females versus definitive males undergoing spermatogonial proliferation. We compared the mapping percentages of these transcriptomes to the two available sea lamprey reference genomes and three annotation files (Ensembl and UCSC for the somatic genome and SIMRbase for the germline genome). We found that mapping the RNA-seq reads to the germline genome gave superior results and, using Trinotate, we provided new putative annotations for 8161 genes in the somatic assembly and 880 genes for the germline assembly. We identified >2000 DEG’s between stages and sexes, as well as biological pathways associated with each. Interestingly, some of the upregulated genes (e.g., DEG’s associated with spermiation) suggest that changes in gene expression can precede morphological changes by several months. In contrast, only 81 DEG’s were evident between the chestnut lamprey (that remains sexually immature during an extended post-metamorphic parasitic feeding phase) and the nonparasitic northern brook lamprey (that undergoes sexual maturation near the end of metamorphosis), but few replicates were available for comparable stages and sexes. This work lays the foundation for identifying and confirming the orthology and the function of genes involved in gonadal development in these and other lamprey species across more developmental stages.