Polymer-based textiles have a major impact on human well-being, as they provide the desired functional protection and aesthetic comfort when worn. However, natural and synthetic polymer-based textiles can also pose serious health risks, as they are surfaces that allow the adhesion of various bacteria, including pathogenic bacteria. To minimise these problems, antibacterial chemical treatments are generally applicable in the case of polymer-based textiles. However, to avoid the use of potentially toxic chemicals, sustainable approaches require the customised design of non-adhesive polymer-based textiles, considering their chemical, physicochemical, constructional, and textural properties. Before designing, several articles are required to gain sufficient knowledge of the described object. Despite the urgent need to combat bacteria (on polymer-based textiles), which pose a serious global health risk, only a few review articles have been published that address bacterial adhesion in the context of superhydrophobic and antibacterial textile materials, while only one review article holistically addresses the textile factors and their influence on this phenomenon. The aim of this review article is to expand the insufficient knowledge about bacterial adhesion to polymer-based textiles on the basis of theoretical findings and real examples through a high degree of structuring, simplification, holistic consideration, and visualization. Therefore, this review provides an insight into the mechanisms involved in bacterial adhesion and a comprehensive overview of the influence of different textile factors, such as chemical composition, hydrophilicity/hydrophobicity, surface charge, surface free energy, roughness, and porosity, on bacterial adhesion. To emphasise the importance of the synergistic effect of the combined textile factors, examples of the influence of hydrophilicity/hydrophobicity in combination with surface charge, surface roughness, and porosity are discussed. From the review, it can be concluded that the combination of hydrophilicity/hydrophobicity and the surface charge of textile fibres and bacteria is crucial for bacterial adhesion, with roughness and porosity being the most important factors among the constructive and textural properties of polymer-based textiles.