Central line-associated bloodstream infections (CLABSI) are mainly preventable with evidence-based guidelines. However, it needs to be made clear how well these guidelines are followed in critical care units in Malaysia. Therefore, this study evaluates critical care nurses’ knowledge, practices, and attitude toward evidence-based guidelines for preventing CLABSIs issued by the Centers for Disease Control and Prevention and the Malaysia Ministry of Health. This study aims to identify the level of knowledge, practice, and attitudes regarding preventing CLABSI among critical care nurses in a teaching hospital. A cross-sectional descriptive study was carried out among 203 nurses working in seven units consisting of the Intensive Care Unit (ICU), Coronary Care Unit (CCU), Intermediate Dependency care Unit (IDCU), Neonatal Intensive Care Unit (NICU), Neurosurgery Intensive Care Unit (Neuro ICU), Cardiac Intensive Care Unit (CICU), and Pediatric Intensive Care Unit (PICU) in University Malaya Medical Center in Kuala Lumpur. Nurses completed online questionnaires regarding their demographic characteristics, knowledge, practices, and attitudes toward evidence-based guidelines for preventing CLABSIs from December 2022 to January 2023. The data was analyzed using T-Test, ANOVA, and Fisher’s Exact Test. 203 ICU nurses completed the questionnaire, and none correctly answered all the knowledge-related questions. The response rate of the correct answer regarding the knowledge question was only 49.7%. Mean scores for 11 questions regarding knowledge of evidence-based guidelines for preventing CLABSI were low, 5.85 (± 1.29). Meanwhile, individual total scores were significantly associated with time spent as a nurse in the ICU (2–4 years). The mean score for eight questions related to CLABSI prevention was 4.19 (± 0.50), which was acceptable for the compliance score. There was an association between nurses’ gender (p-value: 0.01) and their duration of working experiences in the ICU (p-value: 0.04) with the practice regarding the prevention of CLABSI. The overall mean score for nurses’ attitudes toward the guidelines was high; 8.63 (± 1.37). However, there was no association between the demographic characteristic and attitude regarding preventing CLABSI. The knowledge and practice of ICU nurses need to be improved. Regular training with the implementation of available guidelines and protocols was essential to improve the knowledge and compliance on preventing CLABSIs in ICUs.