New network attack platforms such as personal to personal botnets pose a great threat to cyberspace, but there is no corresponding detection method to detect them. In order to improve the security of topological networks, this research designs a mathematical modeling analysis method for potential attack detection based on convolutional neural networks. This method determines the potential attack risk assessment function through the feature extraction of vulnerable areas in network topology and the probability model of potential attacks, and then detects potential attacks by means of convolutional neural network data modeling. The experimental results show that the false detection rate and missed detection rate of the three methods for potential attacks are lower than 9% and 8% respectively, but the false detection rate and missed detection rate of the method given in the study are the lowest, and can always be kept below 5%. At the same time, the detection time of potential attacks of this method is shorter than that of the other two detection methods. The detection of potential attacks provides a technical guarantee for the safe operation of the network.