ObjectivesTo compare the construct validities of the Chinese-versions Critical-Care Pain Observation Tool and Behavioural Pain Scale as measures of critically ill patients’ pain by (a) discriminant validation of behavioural scales and vital signs (e.g. heart rate and mean arterial pressure) during a non-nociceptive procedure (noninvasive blood pressure] assessment) and a nociceptive procedure (endotracheal suctioning), (b) criterion validation of behavioural scales and vital signs with patients’ self-reported pain and (c) testing the interrater reliability of both scores. Research methodology/designIn this crossover, observational study, pain responses of 316 critically ill patients (213 conscious; 103 unconscious) were measured by both the Critical Care Pain Observation Tool and the Behavioural Pain Scale scores, vital signs and self-report (if conscious) during noninvasive blood pressure assessment and endotracheal suctioning procedures. Interrater reliability was tested in nociceptive procedures of a pilot study on 20 critically ill patients. Data were analysed by descriptive statistics, multiple logistic regression analysis and receiver-operating characteristic curves. SettingA medical intensive care unit in a regional teaching hospital in northern Taiwan. ResultsPatients’ self-reported pain was predicted by total Critical Care Pain Observation Toolscores (odds ratio=1.93, p<0.01) and total Behavioural Pain scores (odds ratio=1.83, p<0.01) but not by vital signs after controlling for patients’ demographic and clinical characteristics. Moreover, Chinese-versions had areas under the receiver-operating characteristic curve of 76.4% and 73.1%, respectively, indicating good ability to detect pain. ConclusionsThe Chinese-versions of the Critical care Pain Observation Toll and Behavioural Pain Score have good construct validity and can sensitively discriminate when critically ill patients experience pain or no pain.