This paper presents a new flexible compact multi-modal imaging setup referred to as PEPI (Photon-counting Edge-illumination Phase-contrast imaging) Lab, which is based on the edge-illumination (EI) technique and a chromatic detector. The system enables both X-ray phase-contrast (XPCI) and spectral (XSI) imaging of samples on the centimeter scale. This work conceptually follows all the stages in its realization, from the design to the first imaging results. The setup can be operated in four different modes, i.e. photon-counting/conventional, spectral, double-mask EI, and single-mask EI, whereby the switch to any modality is fast, software controlled, and does not require any hardware modification or lengthy re-alignment procedures. The system specifications, ranging from the X-ray tube features to the mask material and aspect ratio, have been quantitatively studied and optimized through a dedicated Geant4 simulation platform, guiding the choice of the instrumentation. The realization of the imaging setup, both in terms of hardware and control software, is detailed and discussed with a focus on practical/experimental aspects. Flexibility and compactness (66 cm source-to-detector distance in EI) are ensured by dedicated motion stages, whereas spectral capabilities are enabled by the Pixirad-1/Pixie-III detector in combination with a tungsten anode X-ray source operating in the range 40–100 kVp. The stability of the system, when operated in EI, has been verified, and drifts leading to mask misalignment of less than 1 upmum have been measured over a period of 54 h. The first imaging results, one for each modality, demonstrate that the system fulfills its design requirements. Specifically, XSI tomographic images of an iodine-based phantom demonstrate the system’s quantitativeness and sensibility to concentrations in the order of a few mg/ml. Planar XPCI images of a carpenter bee specimen, both in single and double-mask modes, demonstrate that refraction sensitivity (below 0.6 upmurad in double-mask mode) is comparable with other XPCI systems based on microfocus sources. Phase CT capabilities have also been tested on a dedicated plastic phantom, where the phase channel yielded a 15-fold higher signal-to-noise ratio with respect to attenuation.