Medical Speed-of-sound (SoS) imaging, which can characterize medical tissue properties better by quantifying their different SoS, is an effective imaging method compared with conventional B-mode ultrasound imaging. As a commonly used diagnostic instrument, a hand-held array probe features convenient and quick inspection. However, artifacts will occur in the single-angle SoS imaging, resulting in indistinguishable tissue boundaries. In order to build a high-quality SoS image, a number of raw data are needed, which will bring difficulties to data storage and processing. Compressed sensing (CS) theory offers theoretical support to the feasibility that a sparse signal can be rebuilt with random but less sampling data. In this study, we proposed an SoS reconstruction method based on CS theory to process signals obtained from a hand-held linear array probe with a passive reflector positioned on the opposite side. The SoS reconstruction method consists of three parts. Firstly, a sparse transform basis is selected appropriately for a sparse representation of the original signal. Then, considering the mathematical principles of SoS imaging, the ray-length matrix is used as a sparse measurement matrix to observe the original signal, which represents the length of the acoustic propagation path. Finally, the orthogonal matching pursuit algorithm is introduced for image reconstruction. The experimental result of the phantom proves that SoS imaging can clearly distinguish tissues that show similar echogenicity in B-mode ultrasound imaging. The simulation and experimental results show that our proposed method holds promising potential for reconstructing precision SoS images with fewer signal samplings, transmission, and storage.