Metaphors have traditionally been defined as mappings between two conceptual domains, the target and the source. A fundamental question here is how people assign the target and source functions in a given metaphorical expression. The present paper proposes that there are two potential mechanisms, a linguistic one and a conceptual one that people might use to assign the target and source functions in a given metaphorical expression (e.g., “a green inspiration”). The linguistic mechanism is based on certain conventions which assign the target function to certain grammatical categories (e.g., the head noun of a metaphorical noun phrase), and the source function to others (e.g., the modifying adjective). The conceptual preference mechanism favors the assignment of the source function to accessible concepts and the target function to less accessible ones. Since there are clashing cases, namely, cases where the two mechanisms yield conflicting target/source assignments, the question of interest is which of these two mechanisms, the linguistic or the conceptual one is a better predictor of the way people assign the target and source functions in metaphorical expressions. The major goal of the present paper is to investigate this issue for a specific type of figure—the synaesthetic metaphor. The paper begins by introducing a conceptual preference principle, according to which terms belonging to lower sensory modalities (e.g., touch and taste) are generally assigned the source function, while terms belonging to higher sensory modalities (e.g., vision and hearing) are generally assigned the target function. We then introduce a linguistic convention for assigning these functions to the linguistic constituents of the Hebrew noun–noun genitive construction. We report the findings of two interpretation generation experiments which support the hypothesis that the conceptual preference principle overrules the linguistic convention in people’s assignments of the target and source functions in the synaesthetic genitive construction. The implications of these findings are elaborated in the discussion.