Summary Introduction Steganography, the art to hide information inside host media like pictures and movies, and steganalysis, its countermeasure attempting to detect the presence of an hidden information within an innocent-looking document, are frequently reported as promising information security techniques for telemedicine. For the past few years, in the race between image steganography and steganalysis, deep learning has emerged as a very promising alternative to steganalyzer approaches based on rich image models combined with ensemble classifiers. A key knowledge of image steganalyzer, which combines relevant image features and innovative classification procedures, can be deduced by a deep learning approach called convolutional neural networks (CNN). This kind of deep learning networks is so well-suited for classification tasks based on the detection of variations in 2D shapes that it is the state-of-the-art in many image recognition problems. Materials and methods We design a CNN-based steganalyzer for images obtained by applying steganography with a unique embedding key. The proposed CNN has a quite different shape compared to the ones resulting from the earlier works, and it is able to provide high detection accuracy for several steganographics tools when the same stego key is reused during the embedding process. The convolutional part of our proposal starts by a global filtering, using a single filter, followed by a second convolutional layer that produces a reduced set of high-level features (256 features for 512 × 512 pixels input images) thanks to the use of large filters. Results The proposed architecture embeds less convolutions, with much larger filters in the final convolutional layer, and is more general: it is able to deal with larger images and lower payloads. For the “same embedding key” scenario, our proposal outperforms all other steganalyzers, in particular the existing CNN-based ones, and defeats many state-of-the-art image steganography schemes. The information encoded by the final vector of features is so discriminating that the classifier part can be reduced to only two output neurons. We finally evaluated the detection ability of the CNN against two spatial domain steganographic schemes and a frequency domain one. More precisely, we designed a perfect steganalyzer for embedding payloads of 0.4 bit per pixel, and for all the steganographic tools investigated in this article (working either in spatial or in frequency domains). Rather interesting results have been obtained too, albeit to a lesser extent, for a payload value of 0.1 bpp. Discussion and conclusions The obtained results are very encouraging, and they outperform all the previous deep learning proposals for steganalysis. A first step in the design of a universal detector has been achieved too, as we are able to detect HUGO based hidden messages even when a WOW steganographier has been used during the training stage. These results allow us to propose to add fragile watermarks on media like pictures or pdf medical documents, to guarantee the authenticity of the material: any attempt of modification of the support will alter the watermark, proving by doing so the modification. Another application is to add personal and medical information inside medical images.