Biological strategies for bottom-up synthesis of inorganic crystalline and amorphous materials within topographic templates have recently become an attractive approach for fabricating complex synthetic structures. Inspired by these strategies, herein the synthesis of multi-layered, hierarchical inverse colloidal crystal films formed directly on topographically patterned substrates via evaporative deposition, or "co-assembly", of polymeric spheres with a silicate sol-gel precursor solution and subsequent removal of the colloidal template, is described. The response of this growing composite colloid-silica system to artificially imposed 3D spatial constraints of various geometries is systematically studied, and compared with that of direct colloidal crystal assembly on the same template. Substrates designed with arrays of rectangular, triangular, and hexagonal prisms and cylinders are shown to control crystallographic domain nucleation and orientation of the direct and inverse opals. With this bottom-up topographical approach, it is demonstrated that the system can be manipulated to either form large patterned single crystals, or crystals with a fine-tuned extent of disorder, and to nucleate distinct colloidal domains of a defined size, location, and orientation in a wide range of length-scales. The resulting ordered, quasi-ordered, and disordered colloidal crystal films show distinct optical properties. Therefore, this method provides a means of controlling bottom-up synthesis of complex, hierarchical direct and inverse opal structures designed for altering optical properties and increased functionality.