The number of space objects, i.e., active satellites as well as space debris, is growing at an exponential rate. The management of congested space requires networks of sensors to detect and precisely measure the positions of objects in space. We describe a method that improves the positional accuracy of passive optical detection of space objects. The method is based on a laboratory sensor calibration, which is suitable for all types of cameras (CCD and CMOS cameras) and can easily be implemented for existing staring or tracking stations or even used with space-based passive optical sensors. The successful implementation of the method is demonstrated in a test campaign by measuring tracklets of several Earth-orbiting satellites with a ground-based staring system known as APPARILLO. In addition to its use in space object detection, we encourage that the presented method might also inspire other fields of research and technology, where images need to be acquired with precise timing.