This work formally introduces and starts investigating the structure of matrix polynomial algebra extensions of a coefficient algebra by (elementary) matrix-variables over a ground polynomial ring in not necessary commuting variables. These matrix subalgebras of full matrix rings over polynomial rings show up in noncommutative algebraic geometry. We carefully study their (one-sided or bilateral) noetherianity, obtaining a precise lift of the Hilbert Basis Theorem when the ground ring is either a commutative polynomial ring, a free noncommutative polynomial ring or a skew polynomial ring extension by a free commutative term-ordered monoid. We equally address the natural but rather delicate question of recognising which matrix polynomial algebras are Cayley-Hamilton algebras, which are interesting noncommutative algebras arising from the study of $\mathrm{Gl}_{n}$-varieties.