The creation of relief camps following a disaster, conflict or other form of externality often generates additional health problems. The density of people in a highly stressed environment with questionable safe food and water access presents the potential for infectious disease outbreaks. These camps are also not static data events but rather fluctuate in size, composition, and level and quality of service provision. While contextualized geospatial data collection and mapping are vital for understanding the nature of these camps, various challenges, including a lack of data at the required spatial or temporal granularity, as well as the issue of sustainability, can act as major impediments. Here, we present the first steps toward a deep learning-based solution for dynamic mapping using spatial video (SV). We trained a convolutional neural network (CNN) model on a SV dataset collected from Goma, Democratic Republic of Congo (DRC) to identify relief camps from video imagery. We developed a spatial filtering approach to tackle the challenges associated with spatially tagging objects such as the accuracy of global positioning system and positioning of camera. The spatial filtering approach generates smooth surfaces of detection, which can further be used to capture changes in microenvironments by applying techniques such as raster math. The initial results suggest that our model can detect temporary physical dwellings from SV imagery with a high level of precision, recall, and object localization. The spatial filtering approach helps to identify areas with higher concentrations of camps and the web-based tool helps to explore these areas. The longitudinal analysis based on applying raster math on the detection surfaces revealed locations, which had a considerable change in the distribution of tents over space and time. The results lay the groundwork for automated mapping of spatial features from imagery data. We anticipate that this work is the building block for a future combination of SV, object identification and automatic mapping that could provide sustainable data generation possibilities for challenging environments such as relief camps or other informal settlements.