Various environmental building applications use temperature and relative humidity data for controlling and monitoring the user environment to deliver greater comfort to occupants. We present a multi-modal sensor system for delivering temperature and relative humidity data around the user workspace, where each multi-modal sensor is integrated into a luminaire of a connected lighting system. Multiple challenges need to be addressed in such a system: the sensors located at the ceiling need to monitor the remote environment around the user workspace, and the sensor measurements are impacted by thermal variations due to luminaire dimming controls. To address these challenges, fusion methods are presented to process raw sensor measurements to estimate the temperature and relative humidity reliably around the user workspace. The proposed sensing method is evaluated in experimental office setups and shown to meet required levels of application performance.