This paper presents an adaptive Nelder–Mead algorithm to inversely reconstruct the non-uniform strain field from the reflected intensity spectrum of the fiber grating sensor. In this algorithm, the standard operating parameters are modified to adapt to the variable strain profiles, and an improved transfer matrix method is adopted to rapidly calculate the reflected intensity spectrum with a high precision. Thus, the performance of high-dimensional optimization involved in the reconstruction of complex strain distributions is significantly improved. Several numerical examples firstly demonstrate the advantages of the proposed algorithm compared with the standard Nelder–Mead algorithm. Then, experimental investigations are conducted on aluminum alloy plates with central circular hole under tensile and bending loading. A penalty function is added to the objective function to meet the constraints of the optimization variables under experimental conditions. The experimental results are compared with the finite element analysis results, which verifies the effectiveness and accuracy of the proposed algorithm. Finally, we provide some insights about the strain distribution and the resulting spectral distortion. Based on the measured spectrum, a spectral feature parameter (slope of the deflection–intensity ratio) is proposed as an indicator to reveal the size of the hole.