Prospective discrimination of rock instability and fracture is a key problem in mining, slope stability, earthquake triggering, and other research fields. Through the rock fracture experiment, scientists put forward the load unload response ratio (referred to as LURR) to detect the non-linear deformation process of the strain stress curve, judge the degree of rock damage and instability fracture, which is accepted worldwide and has been widely used in seismic risk assessment. But, the extraction of response parameters (including strain, energy, well water level, etc.) are faced with many difficulties in actual observation, which makes the application of results uncertain. In this paper, the change of relative wave velocity is proposed as the loading unloading response parameter. Through rock mechanics experiments, a prospective discrimination method of rock instability and fracture process is constructed. The change characteristics of the LURR in the process of rock instability and fracture under stress are studied, the experiment result show that: when approaching the main fracture, the LURR calculated by taking the transverse strain energy as the response quantity obviously rises and fall back, the acoustic emission energy release rate and event rate are close to exponential acceleration process; The LURR by taking the change of relative wave velocity as the response parameter, first decreases, then fluctuates near zero, and rapidly decreases to below zero when it reaches the critical failure. The change combination of the above parameters’ response ratio can effectively judge the process of rock deformation, instability and fracture under stress. Moreover, this study of the load unload response ratio which takes the change of relative wave velocity as the response parameter, makes up the deficiency of the traditional load unload response ratio method in judging the rock instability of the medium under stress.