Recently, pressure-sensing mats have been widely used to capture static and dynamic pressure over sleep for posture recognition. Both a full-size mat with a low-density sensing array for figuring out the structure of the whole body and a miniature scale mat with a high-density sensing array for identifying the local characteristics around the chest have been investigated. However, both of the mat systems may face the challenge in the trade-off between the computational complexity (involving the size, density, etc. of the mat) and the performance of sleep posture recognition, where high performance may requires overcomplex computation and result in time latency in real-time sleep posture monitoring. In this paper, a lightweight neural network named ConcatNet, is proposed to realize sleep postures (supine, left, right, and prone) recognition in real time while maintaining a favorable performance. In ConcatNet, the inception module is proposed to extract the image features under multiple receptive fields, while the multi-layer feature fusion module is utilized to fuse deep and shallow features to enhance the model performance. To further improve the efficiency of the model, the depthwise convolution is adopoted. ConcatNet models in 3 different scales (ConcatNet-S, ConcatNet-M, and ConcatNet-L) are built to explore the impact of the sensor density on sleep posture recognition performance. Experimental results exhibit that ConcatNet-M corresponding to medium sensor density (16×16) achieved the best comprehensive performance, with short-term data cross-validation accuracy at 95.56% and overnight data testing accuracy at 94.68%. The model size is 7.91KB, FLOPs is 56.47K, and the inference time is only 0.38ms, which shows an outstanding performance of real-time sleep posture recognition with minimum consumption, indicating the potential to be deployed in mobile devices.