Non-Fermi liquids are metals that cannot be adiabatically deformed into free fermion states. We argue for the existence of "non-Fermi glasses," phases of interacting disordered fermions that are fully many-body localized (MBL), yet cannot be deformed into an Anderson insulator without an eigenstate phase transition. We explore the properties of such non-Fermi glasses, focusing on a specific solvable example. At high temperature, non-Fermi glasses have qualitatively similar spectral features to Anderson insulators. We identify a diagnostic based on ratios of correlators that sharply distinguishes between the two phases even at infinite temperature. Our results and diagnostic should generically apply to the high-temperature behavior of MBL descendants of fractionalized phases.