We carried out a detailed investigation on a compact in-line multimode single-mode multimode (MSM) fiber structure. Both theoretical modal and experimental setup were established to demonstrate the transmission characteristics and the corresponding responses of the applied strain and temperature. The proposed structure simply involves a section of the single-mode fiber (SMF) spliced to two sections of multimode fiber (MMF) and lead-in and lead-out SMFs. The excited environment-sensitive cladding modes together with the fundamental mode in the central SMF form a typical Mach–Zehnder interferometer (MZI). We analyzed the transmission characteristics of the different length of the middle SMF and the MMF in detail. In the experiment, we obtained the extinction ratio of the MSM fiber structure based MZI comb spectrum which was up to 20dB, and sensitivities of 0.7096pm/με (0–2000με) and 44.12pm/°C (10–70°C), which proved the potential sensing applications of the proposed fiber structure.