Segmenting articular cartilage and meniscus from magnetic resonance (MR) images is an essential task for the assessment of knee pathology. Most of the previous classification-based works for cartilage and meniscus segmentation only rely on independent labellings by a classifier, but do not consider the spatial context interaction. The labels of most image voxels are actually dependent upon their neighbours. In this study, we present an automatic knee segmentation system working on multi-contrast MR images where a novel classification model unifying an extreme learning machine (ELM)-based association potential and a discriminative random field (DRF)-based interaction potential is proposed. The DRF model introduces spatial dependencies between neighbouring voxels to the independent ELM classification. We exploit a rich set of features From multi-contrast MR images to train the proposed classification model and perform the loopy belief propagation for the inference. The proposed model is evaluated on multi-contrast MR datasets acquired from 11 subjects with results outperforming the independent classifiers in terms of segmentation accuracy of both cartilages and menisci.