InAs quantum dots with InGaAs strain-reducing layer on GaAs(001) grown at three different temperatures were investigated from the aspect of both structural and optical properties. Dislocations originated from the InAs quantum dot (QD) layer were observed at growth temperatures of 490 °C, 500 °C, and 510 °C. Their densities are relatively larger in the cases of 490 °C and 510 °C, where they are caused by strain accumulation at larger-size InAs quantum dots during cover layer growth. Photoluminescence lifetimes at 6 K are almost the same in the three samples. On the other hand, that of the 500 °C-grown sample is an order of magnitude larger than the other two samples at 300 K. This indicates that dislocations act as a non-radiative center to deteriorate optical characteristics. Growth around 500 °C suppresses the growth of larger-size InAs QDs and reduces the InAs strain accumulation, which leads to the dislocation formation at the cover layer.