Using deep learning approaches to identify and locate defects in composite structures made of Carbon Fiber Reinforced Plastics (CFRP) is becoming increasingly popular. However, when the training data is limited, the existing deep learning methods do not work well. Moreover, the lack of data for damaged conditions of CFRP structures causes a severe data imbalance, which makes this problem even harder. In this paper, we present a novel damage detection method called DFMTR which stands for Digital Twin based Few-shot Meta Transfer Learning. It creates numerical model as the digital twin to generate a large amount of virtual data under different damage conditions for model training. Then, it applies relational network and domain adaptive techniques to achieve effective few-shot transfer. That means, it can detect damage accurately with only a very small amount of real data. This is very useful and practical for damage detection of high-end CFRP equipment. In experiments, our approach outperforms existing methods in all commonly used metrics. The classification accuracy can be improved to more than 82% from 55% by using only 4 real samples, and the accurate localization of damaged area is also successfully achieved.