Spatial transcriptomics (ST) technologies have advanced to enable transcriptome-wide gene expression analysis at submicron resolution over large areas. However, analysis of high-resolution ST is often challenged by complex tissue structure, where existing cell segmentation methods struggle due to the irregular cell sizes and shapes, and by the absence of segmentation-free methods scalable to whole-transcriptome analysis. Here we present FICTURE (Factor Inference of Cartographic Transcriptome at Ultra-high REsolution), a segmentation-free spatial factorization method that can handle transcriptome-wide data labeled with billions of submicron-resolution spatial coordinates and is compatible with both sequencing-based and imaging-based ST data. FICTURE uses the multilayered Dirichlet model for stochastic variational inference of pixel-level spatial factors, and is orders of magnitude more efficient than existing methods. FICTURE reveals the microscopic ST architecture for challenging tissues, such as vascular, fibrotic, muscular and lipid-laden areas in real data where previous methods failed. FICTURE's cross-platform generality, scalability and precision make it a powerful tool for exploring high-resolution ST.