Impaired associative memory function in patients with schizophrenia has received considerable attention. However, previous studies have primarily concentrated on unisensory materials, which limits our understanding of the broader implications of this impairment. In this study, we sought to expand on this knowledge by examining two types of associative memory domains in individuals with schizophrenia, leveraging both visual (Vis) and auditory (Aud) materials. A total of 32 patients with schizophrenia and 29 healthy controls were recruited to participate in the study. Each participant participated in an experiment composed of three paradigms in which different abstract materials (Aud-Aud, Aud-Vis, and Vis-Vis) were presented. Subsequently, the discriminability scores of the two groups were calculated and compared in different modal tasks. Results from the study indicated that individuals with schizophrenia demonstrated varying degrees of associative memory dysfunction in both the same and cross-modalities, with the latter having a significantly lower score than healthy controls (t = 4.120, p < 0.001). Additionally, the cross-modal associative memory function was significantly and negatively correlated with the severity of negative symptoms among individuals diagnosed with schizophrenia (r = −0.362, p = 0.042). This study provides evidence of abnormalities in the processing and memorization of information that integrates multiple sensory modalities in individuals with schizophrenia. This is of great significance for further understanding the cognitive symptoms and pathological mechanisms of schizophrenia, potentially guiding the development of relevant interventions and treatment methods.