It is the trend of the development of modern competitive sports to put scientific and technological analysis methods and means into the study of volleyball, and it is also one of the powerful guarantee ways to promote the competitive level of all countries. The random matrix model algorithm has unique advantages to construct the team’s collective technical and tactical ability structure model. The quantitative relationship of the model describes the relationship between the technical and tactical ability structure and the result of victory and defeat and makes the advantages and disadvantages of the team clear, which is conducive to the subsequent targeted training and improvement. The technical and tactical abilities of the teams in different seasons were input to verify the prediction accuracy of the model for the teams in different seasons. In the face of the rapidly changing game situation, the coach team timely transmits the adjusted technical and tactical strategies to the players on the field and deals with the changes accurately and effectively. After the game, the opponent’s strengths and weaknesses should be clarified, and the team’s daily training details should be summarized to provide reference for the cultivation of collective technical and tactical consciousness. The random sample covariance matrix of the random monitoring matrix is constructed and the maximum and minimum eigenvalues of the sample covariance matrix are solved. The ratio of characteristic values is used to construct the detection index of characteristic values, and the detection threshold algorithm of characteristic values is determined to judge the competitive ability of volleyball players. In the case of false alarm rate and matrix size, based on Tracy-Widom distribution characteristics, the maximum eigenvalue and minimum eigenvalue approximations of sample covariance matrix are used to improve the eigenvalue index detection threshold algorithm, and the influence of false alarm rate, matrix size, and other parameters on the improved eigenvalue index detection threshold is further studied. Then, Iris data set was used to verify the effectiveness of the algorithm in terms of accuracy, recall rate, and comprehensive effective value, and the validation results proved that the accuracy of the algorithm reached more than 90%.