Retrieval based on query images supports interesting applications in handwritten document analysis, such as checking manuscripts originality, and authorship. In this respect, writer retrieval systems aim to automatically find all manuscripts belonging to the same author. Presently, we propose a new combination scheme for multiple writer retrieval systems that employ different features and dissimilarities. The proposed combination is founded on writer-independent, SVM dissimilarity learning. For experimental evaluation, three individual systems are proposed each of which, has its specific features. To develop the first system, we propose the Multiscale Histogram Of Templates (M-HOT). For the second system, we introduce the so-called Multi-Gradient Elongated Quinary Pattern (MG-EQP) as new descriptor for handwriting characterization. The third system uses the well-known Run Length Features. Retrieval tests are performed on CVL, ICDAR-2011, ICDAR-2013 and ICDAR-2017 datasets. Furthermore, to highlight the language-independence aspect, experiments are performed on KHATT dataset that contains Arabic handwritten documents. Results obtained evince the effectiveness of the proposed features as well as the combination scheme, which outperforms both individual systems and the state of the art.