Constructing system models in industrial tasks is difficult, especially when multiple sensing information is involved. The peg-in-hole task is typically complex in industrial manufacturing. It is hard to model different assembly states precisely. Learning from demonstration can simplify the modelling process based on human demonstration. In this paper, we transfer a new kind of assembly skill to the robot. To intuitively represent the orientation of the assembly, we propose a novel expression named assembly-angle based on direction cosine and rotation matrix , which offers a good differential performance and avoids the singularity problem. Meanwhile, this approach can be encoded by the dynamic constraint dynamic movement primitives (DC-DMPs) method, which is proposed to consider the constraints among the assembly-angle during movement generalization . The RBF network is employed to generalize force–torque information to improve the accuracy of force–torque profiles prediction. We utilize the vision system and apply admittance control to locate the hole precisely. This proposed assembly skill learning framework can complete the task from random deviation of the pose, which makes it suitable for realistic assembly scenarios. To validate our method’s generalization ability , pegs of two kinds of materials were designed, and the deviations of the initial pose were set from small to large. The success rate was 93.75% for high stiffness assembly and 86.25% for low stiffness assembly. The result validated the effectiveness of our method in the peg-in-hole task. • Human assembly skills can be effectively transferred through the proposed framework. • Assembly-angle has a good generalization performance in the proposed framework. • DC-DMPs considers the dynamical constraints during movement generalization. • RBF network has a high prediction accuracy for forces and torques. • The proposed framework is verified on the aluminum and silicone peg assembly tasks.