Elucidating the structure–property relationship is vital for developing organic luminescent materials with excellent properties and performance optimization. Herein, we designed and synthesized three types of fluorenone-based molecules named FO-Ph, FO-Na and FO-An, with similar structures but different sizes of substituent groups (benzene, naphthalene and anthracene). This slight modification resulted in significantly different optical properties of luminogens in dilute solutions and powders. FO-Ph as an aggregation-caused quenching (ACQ) emitter displayed strong emissions in solution; moreover, after the addition of a poor solution, the fluorescence intensity of FO-Ph gradually decreased. On the contrary, FO-Na displayed aggregation-induced emission enhancement properties, and FO-An was non-emissive in solutions but displayed enhanced emission owing to its molecular aggregation, indicating that FO-An comprises an AIE-active molecule. The characterization of the photophysical properties, theoretical calculations and an X-ray single crystal analysis showed that the emission properties of the three compounds could be affected by the size of substituent groups, twisted geometry and molecular packing modes. The three fluorenone luminogens exhibited different force-induced emission enhancement properties. Moreover, FO-Ph can be used as a water detector and showing high sensitivity. This study elucidates the AIE and ACQ phenomena of fluorenone complexes and provides effective ways for developing organic luminescent materials.