We study a class of matrices with noncommutative entries, which were first considered by Yu.I. Manin in 1988 in relation with quantum group theory. They are defined as “noncommutative endomorphisms” of a polynomial algebra. More explicitly their defining conditions read: (1) elements in the same column commute; (2) commutators of the cross terms are equal: [ M i j , M k l ] = [ M k j , M i l ] (e.g. [ M 11 , M 22 ] = [ M 21 , M 12 ] ). The basic claim is that despite noncommutativity many theorems of linear algebra hold true for Manin matrices in a form identical to that of the commutative case. Moreover in some examples the converse is also true, that is, Manin matrices are the most general class of matrices such that linear algebra holds true for them. The present paper gives a complete list and detailed proofs of algebraic properties of Manin matrices known up to the moment; many of them are new. In particular we provide complete proofs that an inverse to a Manin matrix is again a Manin matrix and for the Schur formula for the determinant of a block matrix; we generalize the noncommutative Cauchy–Binet formulas discovered recently arXiv:0809.3516, which includes the classical Capelli and related identities. We also discuss many other properties, such as the Cramer formula for the inverse matrix, the Cayley–Hamilton theorem, Newton and MacMahon–Wronski identities, Plücker relations, Sylvester's theorem, the Lagrange–Desnanot–Lewis Carroll formula, the Weinstein–Aronszajn formula, some multiplicativity properties for the determinant, relations with quasideterminants, calculation of the determinant via Gauss decomposition, conjugation to the second normal (Frobenius) form, and so on and so forth. Finally several examples and open question are discussed. We refer to [A. Chervov, G. Falqui, Manin matrices and Talalaev's formula, J. Phys. A 41 (2008) 194006; V. Rubtsov, A. Silantiev, D. Talalaev, Manin matrices, elliptic commuting families and characteristic polynomial of quantum gl n elliptic Gaudin model, in press] for some applications in the realm of quantum integrable systems.