Antibacterial drug-loaded electrospun nano- and microfibrous dressings are of major interest as novel topical drug delivery systems in wound care. In this study, chloramphenicol (CAM)-loaded polycaprolactone (PCL) and PCL/poly(ethylene oxide) (PEO) fiber mats were electrospun and characterized in terms of morphology, drug distribution, physicochemical properties, drug release, swelling, cytotoxicity, and antibacterial activity. Computational modeling together with physicochemical analysis helped to elucidate possible interactions between the drug and carrier polymers. Strong interactions between PCL and CAM together with hydrophobicity of the system resulted in much slower drug release compared to the hydrophilic ternary system of PCL/PEO/CAM. Cytotoxicity studies confirmed safety of the fiber mats to murine NIH 3T3 cells. Disc diffusion assay demonstrated that both fast and slow release fiber mats reached effective concentrations and had similar antibacterial activity. A biofilm formation assay revealed that both blank matrices are good substrates for the bacterial attachment and formation of biofilm. Importantly, prolonged release of CAM from drug-loaded fibers helps to avoid biofilm formation onto the dressing and hence avoids the treatment failure.