Purpose: During commissioning of a new linear accelerator (Varian Clinac 21iX), differences were found in dynamic MLC (sliding window) delivery of 2.5–5.5% when compared to an existing matched linear accelerator (Varian Trilogy). An investigation was done to determine the causes of these differences. Method and Materials: Ion chamber and film measurements were done on several accelerators while varying MLC alignment parameters. Test deliveries, including open fields, MLC fields, “sweeping gap”, and dynamic MLC fields were made using a central ion chamber and film. A difference in the software configuration of the MLC (“SureTouch”) resulted in a significant dosimetric effect and measurements were performed to ascertain its impact. Results: Except for dynamic MLC delivery, the accelerators were well matched, with all differences less than 0.5%. Differences in delivery of dynamic IMRT treatments were found in three areas: 1) Calibration of the MLC alignment can introduce up to 2% variation in dMLC performance; 2) A variation of 1.5% appears to be inherent in the mechanical performance of MLCs; 3) A 1% difference appears to be a result of the software configuration of the MLC by service engineers. This configuration can be optionally introduced in cases where poor performance of the MLC is noted during patient deliveries. Conclusion: With proper configuration and calibration of an MLC, machine‐to‐machine performance should be within 2%. As noted in several previous publications, careful calibration and testing of MLC leaf position and alignment are critical to proper delivery of dynamic MLC treatments. Software configuration of the MLC by service engineers can affect dosimetric performance for IMRT deliveries. Medical physicists should monitor MLC configuration changes installed by service personnel carefully, as these changes may require alterations in treatment planning system models.