Phenolic acids including chlorogenic acids are major polyphenolic compounds found in Jerusalem artichoke (Helianthus tuberosus L.). The plant itself is an emerging biorefinery crop due to the inulin-rich tubers, a bioethanol feedstock, but the aerial parts represent a rich source of bioactive compounds. We have determined the level of major phenolic acids in extracts of four plant organs: tuber, leaf, flower, and stem. Employing three heating conditions (20 °C, 60 °C, and microwaving), corrected total phenolic content (TPC) was highest in the leaves (4.5–5.7 mg gallic acid equivalents g−1 dry substance), followed by flower (2.1–2.9), tuber (0.9–1.4), and lowest in stem extracts (0.1–0.2). A previously overlooked interference of the Folin–Ciocalteu assay, namely a signal contribution from ascorbic acid, caused overestimation of TPC in various organs ranging from 65% to 94%. Radical scavenging activity of extracts correlated significantly with TPC, both on corrected (R2 = 0.841) and uncorrected (R2 = 0.884) values. Out of the identified phenolic acids determined by quantitative HPLC-UV analysis, chlorogenic and dicaffeoylquinic acids accounted for 72–82% of corrected TPC in leaf and tuber extracts. Optimization of leaf extraction was tested in a 23-factorial Central Composite Face (CCF) design. Temperature was the most important model term, and a solvent strength of less than 50% ethanol promoted the highest TPC yields. Further developments in extraction processing of crop residues may open avenues for improving the utilization of Jerusalem artichoke in valuable products.