This work is concerned with the study of the extreme rays of the convex cone of \(3\times 3\) quasiconvex quadratic forms (denoted by \(\mathcal{C}_3\)). We characterize quadratic forms \(f\in \mathcal{C}_3,\) the determinant of the acoustic tensor of which is an extremal polynomial, and conjecture/discuss about other cases. We prove that in the case when the determinant of the acoustic tensor of a form \(f\in \mathcal{C}_3\) is an extremal polynomial other than a perfect square, then the form must itself be an extreme ray of \(\mathcal{C}_3;\) when the determinant is a perfect square, then the form is either an extreme ray of \(\mathcal{C}_3\) or polyconvex; finally, when the determinant is identically zero, then the form f must be polyconvex. The zero determinant case plays an important role in the proofs of the other two cases. We also make a conjecture on the extreme rays of \(\mathcal{C}_3,\) and discuss about weak and strong extremals of \(\mathcal{C}_d\) for \(d\ge 3,\) where it turns out that several properties of \(\mathcal{C}_3\) do not hold for \(\mathcal{C}_d\) for \(d>3,\) and thus case \(d=3\) is special. These results recover all previously known results (to our best knowledge) on examples of extreme points of \(\mathcal{C}_3\) that were proved to be such. Our results also improve the ones proven by Harutyunyan and Milton (Commun Pure Appl Math 70(11):2164–2190, 2017) on weak extremals in \(\mathcal{C}_3\) (or extremals in the sense of Milton) introduced in (Commun Pure Appl Math XLIII:63–125, 1990). In the language of positive biquadratic forms, quasiconvex quadratic forms correspond to nonnegative biquadratic forms and the results read as follows: if the determinant of the \({\varvec{y}}\) (or \({\varvec{x}}\)) matrix of a \(3\times 3\) nonnegative biquadratic form in \({\varvec{x}},{\varvec{y}}\in \mathbb R^3\) is an extremal polynomial that is not a perfect square, then the form must be an extreme ray of the convex cone of \(3\times 3\) nonnegative biquadratic forms \((\mathcal{C}_3);\) if the determinant is identically zero, then the form must be a sum of squares; if the determinant is a nonzero perfect square, then the form is either an extreme ray of \(\mathcal{C}_3,\) or is a sum of squares. The proofs are all established by means of several classical results from linear algebra, convex analysis (geometry), real algebraic geometry, and the calculus of variations.