Assembly buildings are an important direction for the future development of the construction field. They can be prefabricated in the factories and then assembled on-site, which significantly improves construction efficiency. The shear walls are the most important lateral force-resisting elements in building structures, and at this stage, there are more and more studies on the prefabricated shear wall. In this paper, a new composite shear wall structure is proposed. The composite shear wall is a part of a prefabricated building, which is prefabricated into a single shear wall unit in the factory. During the construction, the upper and lower prefabricated shear wall units are connected by the plug-in. The design methods of splicing connection are given for the design of this composite shear wall structure. Eleven composite wall models under different parameters are established by using the finite element method, especially the fine modeling of the upper and lower connection parts. Compared with the conventional composite shear wall model of the same dimensions, the mechanical behaviors of the two models are similar. In the simulation of cyclic loading, the new composite shear wall shows good ductility and energy dissipation capacity, and also meets the established requirements of building seismic performance. Therefore, it can be concluded that the new prefabricated composite shear walls have good development prospects and application values.