Canonical duality theory (CDT) is a newly developed, potentially powerful methodological theory which can transfer general multi-scale nonconvex/discrete problems in Rn to a unified convex dual problem in continuous space Rm with m n and without a duality gap. The associated triality theory provides extremality criteria for both global and local optimal solutions, which can be used to develop powerful algorithms for solving general nonconvex variational problems. This thesis, first, presents a detailed study of large deformation problems in 2-D structural system. Based on the canonical duality theory, a canonical dual finite element method is applied to find a global minimization to the general nonconvex optimization problem using a new primal-dual semi-definite programming algorithm. Applications are illustrated by numerical examples with different structural designs and different external loads. Next, a new methodology and algorithm for solving post buckling problems of a large deformed elastic beam is investigated. The total potential energy of this beam is a nonconvex functional, which can be used to model both pre- and post-buckling phenomena. By using the canonical dual finite element method, a new primal-dual semi-definite programming algorithm is presented, which can be used to obtain all possible post-buckled solutions. In order to verify the triality theory, mixed meshes of different dual stress interpolation are applied to obtain the closed dimensions between discretized displacement and discretized stress. Applications are illustrated by several numerical examples with different boundary conditions. We find that the global minimum solution of the nonconvex potential leads to the unbuckled state, and both of these two solutions are numerically stable. However, the local minimum solution leads to an unstable buckled state, which is very sensitive to the external load, thickness of the beam, numerical precision, and the size of finite elements. Finally, a mathematically rigorous and computationally powerful method for solving 3-D topology optimization problems is demonstrated. This method is based on CDT developed by Gao in nonconvex mechanics and global optimization. It shows that the so-called NP-hard Knapsack problem in topology optimization can be solved deterministically in polynomial-time via a canonical penalty-duality (CPD) method to obtain precise global optimal 0-1 density distribution at each volume evolution. The relation between this CPD method and Gao's pure complementary energy principle is revealed for the first time. A CPD algorithm is proposed for 3-D topology optimization of linear elastic structures. Its novelty is demonstrated by benchmark problems. Results show that without using any artificial technique, the CPD method can provide mechanically sound optimal design, also it is much more powerful than the well-known BESO and SIMP methods. Finally, computational complexity and conceptual/mathematical mistakes in topology optimization modeling and popular methods are explicitly…