This paper explores a displacement-based formation control problem in the presence of misaligned orientations among different body coordinate frames. When agents sense and adjust their relative positions, these misalignments suggest the attitudes of the agents in the absence of an agreement, thereby distorting collective behavior. To mitigate this distortion, an angular velocity control protocol is provided and embedded into a positively invariant set, aiming to establish a predefined attitude agreement. Based on this agreement, a formation control protocol is then to achieve the desired formation shape. Unlike the original alignment case, not only the dynamics of the agents among different coordinate frames are considered, but also the connectivity on the underlying topologies is relaxed. It should be pointed out that constructing a common Lyapunov function for these systems is challenging, since typical conditions for its existence and solvability, e.g., the double stochasticity condition or the non-trivial eigenvalue assignment, may not be satisfied. To address these challenges, a geometric analysis framework, derived as several polytopes and associated properties, is extended to handle these systems without the requirement for such conditions. By utilizing these properties, the control objectives of attitude and formation control are achieved, indicating that the desired formation shape is guaranteed through the implementation procedure of the attitude consensus. Finally, an example is conducted to verify the main results.