Purpose: To evaluate the sensitivity of three patient‐specific IMRT QA devices to systematic delivery errors. Methods: Four clinical step‐and‐shoot IMRT plans were selected, two simple prostate, one medium complexity head and neck (H&N) and one complex H&N. Twelve plan variations with errors introduced were created: Multi‐Leaf Collimator (MLC) positional errors (all leaf pairs shifted in the same direction (1, 2, 3 mm) and opposite direction (1, 2, 3 mm)); collimator rotation offsets (1, 2, 3 degrees) and gantry rotation offsets (0.5, 1, 2 degrees). The total dose for each plan was measured using an ArcCHECK helical diode array. Each field for each plan, excluding those with gantry offsets, was measured using an Electronic Portal Imager and a MatriXX Evolution 2D ionisation chamber array. The measured dose was compared to the calculated dose for the no error plan using Gamma analysis with 3%/3mm, 3%/2mm, and 2%/2mm criteria. Results: The ability of the different devices to detect errors was similar. In most cases pass rates reduced with increasing errors and/or stricter gamma criteria as expected. The ArcCHECK demonstrated sensitivity to 2 degree gantry errors, with a reduction in pass rate of 6–14% between the 1 and 2 degree errors in all plans for all gamma criteria. Even for the 2%/2mm criteria, no device detected a 3 degree collimator error in either prostate plan, however collimator errors were detected for the H&N plans which had elongated fields. For all criteria, there were different scenarios for each detector where MLC position errors of 3 mm were not detected. Conclusion: None of the devices could reliably detect MLC position or collimator rotation errors that were not within recommended linear accelerator monthly QA tolerances. There were no consistent differences between the ability of the devices to detect systematic delivery errors. Financial support has been provided by the Saudi Arabian Cultural Mission towards this work.