At CRYPTO 2019, A. Gohr made a breakthrough in combining classical cryptanalysis and deep learning and applied his method to round reduced SPECK successfully. However, his suggested neural-based distinguisher scheme is only limited to differential cryptanalysis. In this paper, we have the following contributions: 1. We combine integral cryptanalysis and deep learning to propose our neural-based integral distinguisher scheme for the first time. To illustrate the effectiveness of our distinguisher scheme, we apply it to block ciphers of different structures, such as substitution-permutation structure ciphers (PRESENT and RECTANGLE), Feistel structure cipher (LBLOCK), and add-rotate-XOR cipher (SPECK) and compare the results with the state-of-the-art classical integral distinguishing method, namely, the bit-based division property. To our great surprise, our neural network-based integral distinguisher can extend the number of distinguished rounds for all block ciphers by two additional rounds (except RECTANGLE, where it is improved by one round) under the same data complexity. 2. As an additional advantage of our scheme, we demonstrate that our Neural Distinguisher (ND) is not only helpful for block cipher designers but also can assist attackers to mount key recovery attacks. To this end, we show how to exploit our ND to mount a key recovery attack and apply it to SPECK32/64. Out of the 1000 trials of key recovery attacks with different keys in 45% of cases, the first suggested subkey is exactly the real subkey of the last round of the cipher. For the remaining 55%, the second or third suggested subkey is exactly the real subkey of the last round of the cipher. 3. To have a piece of concrete evidence for the advantage of our scheme over classical integral methods, we design an experiment known as the same-difference experiment. In this experiment, we show that our ND can learn some features beyond the capabilities of classical integral methods. We then propose a set of features that can justify the gap between classical integral methods and our neural-based integral distinguisher and verify them by further experiments.