Validated tools that measure quality of life (QOL) for children with poor prognosis malignancies are not available. We are developing a novel instrument, The Pediatric Advanced Care-Quality of Life Scale (PAC-QoL), in order to address this gap. Instrument development requires a phase of item reduction and assessment of item comprehension in the target population. This manuscript provides a report on this phase in the development of the PAC-QoL. Children with poor prognosis cancer and/or their parents were invited to participate in cognitive probing interviews. Participants' understanding of each item was rated from 0 (did not understand) to 4 (completely understood). To evaluate the response scale, an overall percentage of respondents' ability to accurately distinguish between the four response options was calculated. Four age- and reporter-specific versions of the PAC-QoL were tested with 74 participants. Mean (±SD) comprehension scores across versions ranged from 3.40 ± .0.30 (child self-report) to 3.69 ± 0.23 (parent of toddler report). The number of items deleted or modified to improve understandability ranged from 46% of all items on the parent-of-child report to 56% for the child and adolescent self-reports. Respondent's abilities to accurately distinguish between response-scale options ranged from 84% (child-report) to 98% (parent-toddler report). We demonstrate a high degree item understandability and response-scale separation in the current version of the PAC-QoL. The scale is ready for psychometric evaluation in its target population.