Biological materials found in nacre or glass sponges reveal specific layered organic and inorganic structures known for their fracture toughness caused by crack deflection and −bridging. This work aims to bio-mimic the brick-and-mortar (BnM) structure from nacre and the layer-by-layer (LbL) structure from the glass sponge filaments to incorporate these effects into the contact zone of carbon fiber-reinforced concrete (CFRC) in order to prevent brittle composite failure. To build up BnM- and LbL-structure materials such as nanoclays, sodium water–glass and polymer dispersions were selected since they are well-established low-cost materials in building. Nanoclays were analyzed regarding their size, dispersibility and exfoliation. Montmorillonite (MMT) was used to be mixed with polymers to produce self-assembled BnM-structured films and coatings. Also, LbL structures were formed by alternating layers of sodium water glass and polymer. Scanning electron microscopy and energy-dispersive X-ray spectroscopy were used to verify the morphology. The MMT-containing coatings demonstrated enhanced nucleation potential when exposed to cementitious eluate. Micromechanical pull-out tests on single carbon fibers with BnM- and LbL-coatings embedded in concrete demonstrate the potential to increase composite toughness. The successful implementation of the bio-inspired structures using affordable materials lays the groundwork for their scalability and integration into composite structures for building.