A novel variable stiffness optimization (VSO) algorithm is proposed accounting for the inherent mechanical characteristics of variable angle tow (VAT) plates. Through sampling in the design space of lamination parameters (LPs), a good initial point is identified in stacking sequence design domain. The concept of tailoring stiffness point by point through-thickness is then introduced. By using the linear fiber path function, design variables for VAT plates are divided into three groups, and then a layerwise optimization approach (LOA) is applied to optimize the angles from the outermost to the innermost stacking position within the three groups, sequentially and iteratively. Through a point-by-point optimization of fiber paths, the stiffness of VAT plates is enhanced part by part. Ritz-type buckling solutions are presented for VAT composite plates with arbitrary boundary conditions. Using first-order shear deformation theory (FSDT), the translational and rotational degrees of freedom of the plate are divided into in-plane and out-of-plane components to solve the prebuckling and buckling problems independently. Buckling loads of VAT plates are optimized under multiple boundary conditions and loadings. Results indicate that VSO algorithm is efficient and robust, and the buckling resistance capacity of optimized VAT plates can be improved approximately twice over constant-stiffness laminates.