Understanding the surrounding environment is an essential part of intelligent autonomous platform to complete autonomous driving or other autonomous tasks. Thereinto, scene understanding is a key technology to understand environmental information. For intelligent autonomous platform, it not only requires high accuracy, but also has a strict requirement for the inference speed. Moreover, it should be able to perform normally in complex environments such as target occlusion, the influence of illumination, and objects at different sizes. For tackling these problems, in this study, a lightweight multi-scale feature dense cascade neural network (LMFDCNet) is proposed in real-time scene understanding in complex scenes. Furthermore, trilateral fusion module is adopted to enhance low-level detailed features or high-level semantic features of each branch to describe the object better in different environments. A multi-scale upsampling module that fuses contextual information at different scales allows objects to be represented at different size. The results indicate that LMFDCNet produces 99.1 % classification accuracy on the tobacco dataset. On the Cityscapes dataset, we achieve 74.2 % Mean Intersection over Union (MIoU) with 13.6 M parameters at the speed of 38.2 FPS on a single 1070Ti card. To test the performance on the real-time application in the real world, LMFDCNet is deployed on the autonomous platform for real-time vision task such as classification of tobacco leaf state in curing process and semantic segmentation in the driving process of autonomous platform. The results reveal that LMFDCNet perform well on the AI embedded device and could be applied into intelligent autonomous platform.