This paper establishes the notion and properties of angle rigidity for 3D multi-point frameworks with angle constraints, and then designs direction-only control laws to stabilize angle rigid formations of mobile agents in 3D. Angles are defined using the interior angles of triangles within the given framework, which are independent of the choice of coordinate frames and can be conveniently measured using monocular cameras and direction-finding arrays. We show that 3D angle rigidity is a local property, which is in contrast to the 3D bearing rigidity as has been proved to be a global property in the literature. We demonstrate that such angle rigid and globally angle rigid frameworks can be constructed through adding repeatedly new points to the original small angle rigid framework with carefully chosen angle constraints. We also investigate how to merge two 3D angle rigid frameworks by connecting three points of one angle rigid framework simultaneously to the other. When angle constraints are given only in the surface of a framework, angle rigidity of convex polyhedra is studied, in which the cases of triangular face and triangulated face are considered, respectively. The proposed 3D angle rigidity theory is then utilized to design decentralized formation control strategies using local direction measurements for teams of mobile agents. Simulation examples are provided to validate the convergence of the formations.