This study aims to develop an innovative image-based 3D modeling-to-simulation framework that can efficiently assess the hazard vulnerability of single-wythe masonry structures. Using the novel concept of ‘reverse descriptive geometry’, the framework utilizes 2D masonry wall images to model a 3D single-wythe masonry structure, subsequently converting it into a 3D discrete element model for the vulnerability assessment. The methodology involves segmenting all bricks in the 2D images, identifying the brick geometries by tracing the boundaries, and approximating those into polygons using the splitting method. By incorporating the wall orientations, the simplified 2D polygons are introduced into 3D space and subsequently extruded to a specified thickness. The corners between the walls, i.e., adjoining wythes, are realistically modeled with a type of structural masonry bond. The digitally constructed 3D masonry geometry is enriched with physical properties such as brick mass and mortar strengths, enabling a seamless 3D discrete element simulation to gauge the hazard vulnerability. The proposed framework is successfully applied to Stylite Tower, a heritage single-wythe masonry structure in Jordan. The results demonstrate a remarkable geometric accuracy achieved and further illustrate the structure's potential vulnerability under seismic events, highlighting the efficiency and efficacy of the developed framework. This study introduces a streamlined procedure leveraging a single 2D image per wall for 3D modeling of a structure, thus highly efficient. A discrete element modeling is adopted instead of finite element methods based on a continuum theory. Therefore, this approach accurately captures the discontinuous nature of masonry structures, resulting in a high fidelity in simulation. Furthermore, this study adopts an impulse-based dynamic simulation method that is far more computationally efficient and tolerant to geometric errors than the conventional discrete element method. Therefore, the proposed framework presents a significant advancement in the vulnerability assessment of masonry structures, providing a novel, efficient, and reliable methodology for engineering practice.