Let g be a complex simple Lie algebra and Z(g) be the center of the universal enveloping algebra U(g). Denote by Vλ the finite-dimensional irreducible g-module with highest weight λ. Lehrer and Zhang defined the notion of strongly multiplicity free representations for simple Lie algebras motivated by studying the structure of the endomorphism algebra EndU(g)(Vλ⊗r) in terms of the quotients of the Kohno's infinitesimal braid algebra. Kostant introduced the g-invariant endomorphism algebras Rλ(g)=(EndVλ⊗U(g))g and Rλ,π(g)=(EndVλ⊗π(U(g)))g. In this paper, we give some other criteria for a multiplicity free representation to be strongly multiplicity free by classifying the pairs (g,Vλ), which are multiplicity free and for such pairs, Rλ(g) and Rλ,π(g) are generated by generalizations of the quadratic Casimir elements of Z(g).