Medical images play an important role in clinics. In most clinic sites, the diagnosis of diseases and the comprehending of disease progression need firstly accurate interpretation of the available medical images, which would be time-consuming in manual interpretation of the accumulated large amount of medical images. Thus automatical analysis and understanding of the available medical images become an active research topic, and therein, feature extraction of medical images plays an important role for achieving diagnosis performance. Natural images hold two-dimensional structures and linear statistical methods, such as k-means, GMM, and sparse coding, are widely applied for extracting compact and inherent representations. In contrast, medical images themselves have three-dimensional structures and possibly consist of multi-phase extension. Directly applying linear methods on the available medical data would lead to high dimensional vectors by reshaping the multi-dimensional data and would destroy the correlated relation among different dimensions of the raw medical data domain. Therefore, this study proposes a multilinear extension of the linear sparse coding for extracting compact and effective intermediate representations of the multi-dimensional local structures in multi-phase CT images, and aggregating the intermediate representations in the Bag-of-Visual-Words (BoVW) manner for classification of focal liver lesions (FLLs). In the proposed approach, three-layer volumes from the corresponding slices of multi-phase CT images are formed and spatiotemporal local structures from the volumes are extracted as 3rd-order tensors. Regarding the high dimensional local structures as tensors, we propose a K-CP (CANDECOMP/PARAFAC) algorithm to learn a tensor dictionary in an iterative way and extract the sparse representation with a multilinear OMP method. The aggregation of the sparse representation is implemented in the BoVW manner, which has been proved to be an effective method for extracting features from natural images and medical images. The proposed strategy is evaluated in classification of focal liver lesions and achieved better results than conventional BoVW with linear statistical methods.