Childhood adversity is associated with various clinical dimensions in psychosis; however, how genetic vulnerability shapes the adversity-associated psychopathological signature is yet to be studied. We studied data of 583 First Episode Psychosis (FEP) cases from the EU-GEI FEP case-control study, including Polygenic risk scores for major depressive disorder (MDD-PRS), bipolar disorder (BD-PRS) and schizophrenia (SZ-PRS); childhood adversity measured with the total score of the Childhood Trauma Questionnaire (CTQ); and positive, negative, depressive and manic psychopathological domains from a factor model of transdiagnostic dimensions. Genes and environment interactions were explored as a departure from a multiplicative effect of PRSs and total CTQ on each dimension. Analyses were adjusted for age, sex, 10 PCA, site of recruitment and for medication. A childhood adversity and PRS multiplicative interaction was observed between A) the CTQ and MDD-PRS on the predominance of positive (β = 0.42, 95% CI = [0.155, 0.682], p = 0.004); and depressive (β = 0.33, 95% CI = [0.071, 0.591], p = 0.013) dimensions; B) between the CTQ and BD-PRS on the positive dimension (β = 0.45, 95% CI = [0.106, 0.798], p = 0.010), and C) with the CTQ and SZ-PRS on the positive dimension (β = −0.34, 95% CI = [−0.660, −0.015], p = 0.040). Bonferroni corrected p-value of significance was set at 0.0125. In conclusion, despite being underpowered, this study suggests that genetic liability for MDD and BD may have a moderating effect on the sensibility of childhood adversity on depressive and positive psychotic dimensions. This supports the hypothesis of an affective pathway to psychosis in those exposed to childhood adversity.