Only 30% of embryos from in vitro fertilized oocytes successfully implant and develop to term, leading to repeated transfer cycles. To reduce time-to-pregnancy and stress for patients, there is a need for a diagnostic tool to better select embryos and oocytes based on their physiology. The current standard employs brightfield imaging, which provides limited physiological information. Here, we introduce METAPHOR: Metabolic Evaluation through Phasor-based Hyperspectral Imaging and Organelle Recognition. This non-invasive, label-free imaging method combines two-photon illumination and AI to deliver the metabolic profile of embryos and oocytes based on intrinsic autofluorescence signals. We used it to classify i) mouse blastocysts cultured under standard conditions or with depletion of selected metabolites (glucose, pyruvate, lactate); and ii) oocytes from young and old mouse females, or in vitro-aged oocytes. The imaging process was safe for blastocysts and oocytes. The METAPHOR classification of control vs. metabolites-depleted embryos reached an area under the ROC curve (AUC) of 93.7%, compared to 51% achieved for human grading using brightfield imaging. The binary classification of young vs. old/in vitro-aged oocytes and their blastulation prediction using METAPHOR reached an AUC of 96.2% and 82.2%, respectively. Finally, organelle recognition and segmentation based on the flavin adenine dinucleotide signal revealed that quantification of mitochondria size and distribution can be used as a biomarker to classify oocytes and embryos. The performance and safety of the method highlight the accuracy of noninvasive metabolic imaging as a complementary approach to evaluate oocytes and embryos based on their physiology.