The connectome is regarded as the key to brain function in health and disease. Structural and functional neuroimaging enables us to measure brain connectivity in the living human brain. The field of connectomics describes the connectome as a mathematical graph with its connection strengths being represented by connectivity matrices. Graph theory algorithms are used to assess the integrity of the graph as a whole and to reveal brain network biomarkers for brain diseases; however, the faulty wiring of single connections or subnetworks as the structural correlate for neurological or mental diseases remains elusive. We describe a novel approach to represent the knowledge of human brain connectivity by a semantic network – a formalism frequently used in knowledge management to describe the semantic relations between objects. In our novel approach, objects are brain areas and connectivity is modeled as semantic relations among them. The semantic network turns the graph of the connectome into an explicit knowledge base about which brain areas are interconnected. Moreover, this approach can semantically enrich the measured connectivity of an individual subject by the semantic context from ontologies, brain atlases and molecular biological databases. Integrating all measurements and facts into one unified feature space enables cross-modal comparisons and analyses. We used a query mechanism for semantic networks to extract functional, structural and transcriptome networks. We found that in general higher structural and functional connectivity go along with a lower differential gene expression among connected brain areas; however, subcortical motor areas and limbic structures turned out to have a localized high differential gene expression while being strongly connected. In an additional explorative use case, we could show a localized high availability of fkbp5, gmeb1, and gmeb2 genes at a connection hub of temporo-limbic brain networks. Fkbp5 is known for having a role in stress-related psychiatric disorders, while gmeb1 and gmeb2 encode for modulator proteins of the glucocorticoid receptor, a key receptor in the hormonal stress system. Semantic networks tremendously ease working with multimodal neuroimaging and neurogenetics data and may reveal relevant coincidences between transcriptome and connectome networks.