Chemical short-range order (CSRO) was assumed as one of the most important structure feature of high entropy alloys and the influence of CSRO on mechanical properties is a fundamental issue yet to be fully understood. In this work, we performed extensively nanoindentation experiments on CoCrFeNiAlx alloys to study the effect of CSRO on the incipient nanomechanical properties. The statistical nature of strengths at the first pop-in event was analyzed to gain insight of deformation mechanisms. All samples examined here exhibit bimodal distribution which indicate non-unique dislocation nucleation mechanisms. The bimodal distributions can be decomposed into two Gaussian distributions and the activation volumes can be obtained in the range of 0.73–1.38b3. The peaks shift to higher stress level after the development of CSRO. The heterogeneous dislocation nucleation plays a dominant role at low indentation stress with the aid of pre-existing crystalline defects. The homogeneous dislocation nucleation mechanism prevails when indentation stress close to theoretical values. The transmission electron microscopy characterization indicates the presence of chemical ordering in the aged samples. Both the degree of chemical ordering and lattice distortion are much higher in the Al containing HEAs due to the distinctive difference of properties in Al and other transition element atoms.