This paper is the first part of a two part paper which introduces a method for determining the vanishing topology of nonisolated matrix singularities. A foundation for this is the introduction in this first part of a method for obtaining new classes of free divisors from representations V of connected solvable linear algebraic groups G. For equidimensional representations where dimG = dimV , with V having an open orbit, we give sufficient conditions that the complement E of this open orbit, the “exceptional orbit variety”, is a free divisor (or a slightly weaker free* divisor). We do so by introducing the notion of a “block representation”which is especially suited for both solvable groups and extensions of reductive groups by them. This is a representation for which the matrix representing a basis of associated vector fields on V defined by the representation can be expressed using a basis for V as a block triangular matrix, with the blocks satisfying certain nonsingularity conditions. We use the Lie algebra structure of G to identify the blocks, the singular structure, and a defining equation for E. This construction naturally fits within the framework of towers of Lie groups and representations yielding a tower of free divisors which allows us to inductively represent the variety of singular matrices as fitting between two free divisors. We specifically apply this to spaces of matrices including m×m symmetric, skew-symmetric or general matrices, where we prove that both the classical Cholesky factorization of matrices and a further “modified Cholesky factorization”which we introduce are given by block representations of solvable group actions. For skew-symmetric matrices, we further introduce an extension of the method valid for a representation of a nonlinear infinite dimensional solvable Lie algebras. In part II, we shall use these geometric decompositions and results for computing the vanishing topology for nonlinear sections of free divisors to compute the vanishing topology for matrix singularities for all of the classes.