Complexity measures typically represented by entropy are capable of detecting and characterizing underlying dynamic changes in a system, and they have been considerably studied for machine condition monitoring and fault diagnosis. Various entropies have been developed based on Shannon entropy to meet actual demands. Nevertheless, currently existing research works about complexity measures mainly focus on experimental studies, and their theoretical studies are still ongoing and not fully explored. In previous studies, it was theoretically and experimentally proved that two complexity measures including correlation dimension and approximate entropy have a ‘bilateral reduction’ effect. Since sample entropy and fuzzy entropy are two more advanced complexity measures that were developed based on the concept of correlation dimension and approximate entropy, this paper continues conducting theoretical and experimental investigations on sample entropy and fuzzy entropy and exploring their theoretical properties to enrich the domain of complexity measure analysis and its applications to machine condition monitoring. Specifically, this paper theoretically proves and verifies that sample entropy and fuzzy entropy still have a similar ‘bilateral reduction’ effect with correlation dimension and approximate entropy, and they are indeed complexity measures. The relationships between sample entropy, fuzzy entropy, and their key parameters during their calculation are numerically and experimentally studied. Bearing and gear run-to-failure datasets are used to investigate the effectiveness of sample entropy and fuzzy entropy for bearing and gear condition monitoring, and experimental results of sample entropy and fuzzy entropy are well-matched with the theoretical ‘bilateral reduction’ effect of sample entropy and fuzzy entropy. Overall, this paper will provide a guideline for correct uses of sample entropy and fuzzy entropy for engineering applications, especially for machine condition monitoring.