The present research focused on the extraction optimization of bioactive compounds from coffee silverskin (CS), a by-product generated in large amounts worldwide during the coffee roasting process. The effect of the different extraction conditions has been investigated by the exploitation of the response surface methodology (RSM). The antioxidant activity assays, such as ABTS and FRAP, total phenolics content (TPC), browning index (BI), and chromatographic analysis of caffeine and chlorogenic acids contents have been performed to evaluate the CS extracts characteristics. The most favorable extraction conditions on the maximum recovery of antioxidant bioactive compounds were found to be as follows: 50% aqueous solution of ethanol (v/v) in solvent solid ratio of 45 mL/g CS, during 30 min at 60 °C. The CS extract prepared in this extraction variant reached the values for ABTS, FRAP, TPC, and BI approximately 101.6 µmol Trolox/g d.m. CS, 132.3 µmol Fe (II)/g d.m. CS, 52.3 µmol GAE (gallic acid equivalent) per g d.m. CS, and 0.3, respectively. Additionally, this extract is rich in caffeine (6 mg/g d.m. CS) and chlorogenic acids (0.22 mg/g d.m. CS). In conclusion, this research demonstrates that CS could be considered as a valuable by-product of bioactive compounds with potential applications in the food industry.