Human neutrophil elastase (HNE) is involved in a number of essential physiological processes and has been identified as a potential therapeutic target for treating acute and chronic lung injury. Nevertheless, only one drug, Sivelestat, has been approved for clinical use and just in Japan and the Republic of Korea. Thus, there is an urgent need for the development of low-molecular-weight synthetic HNE inhibitors, and we have developed a wide variety of HNE inhibitors with various chemical scaffolds. We hypothesized that substitution of the active fragment of Sivelestat into these HNE inhibitor scaffolds could modulate their inhibitory activity, potentially resulting in higher efficacy and/or improved chemical stability. Here, we report the synthesis, biological evaluation, and molecular modeling studies of novel compounds substituted with the 4-(sulfamoyl)phenyl pivalate fragment necessary for Sivelestat activity. Many of these compounds were potent HNE inhibitors with activity in the nanomolar range (IC50 = 19–30 nM for compounds 3a, 3b, 3f, 3g, and 9a), confirming that the 4-(sulfamoyl)phenyl pivalate fragment could substitute for the N-CO group at position 1 and offer a different point of attack for Ser195. Results of molecular docking of the these pivaloyl-containing compounds into the HNE binding site supported the mechanism of inhibitory activity involving a nucleophilic attack of Ser195 from the catalytic triad onto the pivaloyl carbonyl group. Furthermore, some compounds (e.g., 3a and 3f) had a relatively good stability in aqueous buffer (t1/2 > 9 h). Thus, this novel approach led to the identification of a number of potent HNE inhibitors that could be used as leads for the further development of new therapeutics.