Numerical modelling of complex structural discontinuities such as fractures poses a computational challenge, as it involves solving multi-scale and multi-physics phenomena and simulating various processes, including solid, fluid, thermal and chemical interactions. To overcome the limitations of long computation times, simplifications or conceptualizations are often required. However, in multi-physics modelling, it is desirable to obtain predictions of certain parameters without making simplifications. In this study, a data-driven deep learning approach is presented that predicts physical permeability parameters through discontinuities with complicated geometries based on digital images. Images of fractures were generated from a digitalized rough fracture surface of subsurface rock. Permeability was calculated using the Stokes equation and Finite Volume discretization for training and testing purposes. Two cases were analyzed: when the fluid velocity field of the fracture was provided to the CNN for training, and a more challenging case when it was not. Results show that deep learning can accurately predict permeability without fluid velocity information. Besides, the model generalizes well, providing accurate predictions of permeability for fractures with significantly different roughness parameters. In conclusion, this approach can reduce computation time during multi-physics modelling and can be used to predict continuous physical permeability values from an image of a fracture with a complex surface.