For the time being, smartphone devices rely on direct interaction from the users for unlocking and authentication purposes through implicit authentication systems such as PINs, facial recognition or fingerprint scanning. While different passive two-factor authentication systems based on machine learning were explored in recent work, all require an implicit authentication system. In this study, the focus is to develop and introduce a passive authentication system based on walking patterns. In this scenario, the authentication system continuously authenticates the user in the background, without any further action. To the best of our knowledge, this is the first study in which the data sets are processed with the aim to generate better performing gait-based motion signals. Compared to previous studied work, we employ a processing stage in which we extract tiny frames of data from the motion signals. Our contribution of processing gait data, allows for more robust learning of the subject movement and lowers the number of samples required to classify a user thereafter. Hence, our approach is more robust compared to using raw gait signals. Further, we transform them into gray-scale images for deep neural network training and feature extraction. Conducting the experiments, the empirical results demonstrate that subjects can be identified with a very high accuracy through walking patterns employing the presented techniques. Empirical results outline that a system based on gait data can be utilized as a passive authentication system. Therefore, it is concluded that deep neural networks employing the technique described in this work for gait-based feature representation are well suited for continuous and unobtrusive authentication systems.