The diagnosis of Coronavirus Disease 2019 (COVID-19) exploiting machine learning algorithms based on chest computed tomography (CT) images has become an important technology. Though many excellent computer-aided methods leveraging CT images have been designed, they do not possess sufficiently high recognition accuracy. Besides, these methods entail vast amounts of training data, which might be difficult to be satisfied in some real-world applications. To address these two issues, this paper proposes a novel COVID-19 recognition system based on CT images, which has high recognition accuracy, while only requiring a small amount of training data. Specifically, the system possesses the following three improvements: 1) Data: a novel redesigned BCELoss that incorporates Label Smoothing, Focal Loss, and Label Weighting Regularization (LSFLLW-R) technique for optimizing the solution space and preventing overfitting, 2) Model: a backbone network processed by two-phase contrastive self-supervised learning for classifying multiple labels, and 3) Method: a decision-fusing ensemble learning method for getting a more stable system, with balanced metric values. Our proposed system is evaluated on the small-scale expanded COVID-CT dataset, achieving an accuracy of 94.3%, a precision of 94.1%, a recall (sensitivity) of 93.4%, an F1-score of 94.7%, and an Area Under the Curve (AUC) of 98.9%, for COVID-19 diagnosis, respectively. These experimental results verify that our system can not only identify pathological locations effectively, but also achieve better performance in terms of accuracy, generalizability, and stability, compared with several other state-of-the-art COVID-19 diagnosis methods.