We study algebraic structures of certain submonoids of the monoid of homology cylinders over a surface and the homology cobordism groups, using Reidemeister torsion with non-commutative coefficients. The submonoids consist of ones whose natural inclusion maps from the boundary surfaces induce isomorphisms on higher solvable quotients of the fundamental groups. We show that for a surface whose first Betti number is positive, the homology cobordism groups are other enlargements of the mapping class group of the surface than that of ordinary homology cylinders. Furthermore we show that for a surface with boundary whose first Betti number is positive, the submonoids consisting of irreducible ones as 3-manifolds trivially acting on the solvable quotients of the surface group are not finitely generated.