Thermal imaging cameras are expensive, particularly those designed for measuring high temperature objects with low measurement uncertainty. A wide range of research and industrial applications would benefit from lower cost temperature imaging sensors with improved metrology. To address this problem, we present the first ever quantification methodology for the temperature measurement performance of an ultra-low cost thermal imaging system based on a smartphone sensor. The camera was formed from a back illuminated silicon Complementary Metal Oxide Semiconductor (CMOS) sensor, developed for the smartphone camera market. It was packaged for use with a Raspberry Pi computer. We designed and fitted a custom-made triplet lens assembly. The system performance was characterised with a range of state-of-the-art techniques and metrics: establishing a temperature resolution of below 10 °C in the range 600–1000 °C. Furthermore, the scene dependent aspects of combined uncertainty were considered. The minimum angular subtense for which an accurate thermal measurement could be made was determined to be 1.35°, which corresponds to a 23 mm bar at a distance of 1 m, or 45:1 field-of-view in radiation thermometer nomenclature.