One challenge of using X-ray computed tomography (CT) is the detection limit of the smallest feature. This is important for self-piercing riveting joints because it sets a limit on the defect size that is detectable with the technology. To address this challenge, defined size notches were generated on the surface of the rivet leg using a focused-ion beam. Five different X-ray CT parameters: distance between the X-ray source, integration time, number of integrations, number of projected images and X-ray CT voltage, were investigated. The area of the notches was measured from the X-ray CT images, which was converted into probability of detection responses. The results show that the most significant parameter is the voxel size determined by the distance between the X-ray CT source and the sample. At shorter distances, the X-ray CT can capture more detail about the defects. This factor also drives the accuracy of the area evaluation and the detected shape of the notches. In addition, the increase in scan duration can be used to further enhance the defect detectability and measurement accuracy of the X-ray CT. It was found that the smallest defects that were able to be detected were 2–4 times the voxel size whilst for the defects to be accurately measured they need to be 3–5 times the voxel size.