A broadband high-resolution real-time four-dimensional imaging spectrometer (HRRFDIS) is presented, which can acquire both broadband fine spectra and high-resolution three-dimensional (3D) spatial images of a 3D object in real time. The HRRFDIS consists of a first microlens array arranged in a plane to achieve orthographic view spatial imaging, a second microlens array arranged on a conical surface to measure the depth and to achieve 360-degree side-view spatial imaging, multiple optical fibers, a collimating microlens array arranged in a straight line, a parallel planar transmission grating pair to obtain high spectral resolution over a broadband spectral range, and an area-array detector. Compared with the scanning four-dimensional imaging spectrometer (FDIS), the HRRFDIS can obtain a broadband high-resolution four-dimensional dataset using only one frame of data, and it is more stable, compact, small-sized, and lightweight. Compared to the staring FDIS using a liquid crystal filter and requiring at least one modulation period of liquid crystal to acquire a complete hyperspectral image, the HRRFDIS can acquire a complete broadband hyperspectral image in real time. Compared to existing snapshot FDIS, the HRRFDIS can achieve much higher spectral resolution, especially over a broadband spectral range. The HRRFDIS is a unique concept that is the first to obtain both high-resolution broadband spectral information and high-resolution 3D spatial information in real time, to the best of our knowledge. The HRRFDIS will be suitable for real-time measurement of 3D objects in the ultraviolet to infrared spectral range.