State-of-the-art computational design approaches for designing the assembly of a structure often rely on high-precision building objects, which can be difficult and costly to fabricate. To consider a wider range of construction materials and fabrication processes, e.g. low-cost 3D printing and brick making, this research introduces a novel computational design method for fabrication-aware assembly problems using existing building components with dimensional variations. Given a finite set of building objects, this study develops a statistical preprocessing approach and a modified pattern search with an efficient integer linear programming method to generate the assembly of a target structure, such that it is statically stable and optimized for geometric smoothness and space utilization. In two case studies, the proposed algorithm exhibits significantly better boundary fitting and space utilization compared with a genetic algorithm and a standard pattern search method and lower computational cost compared with a genetic algorithm.