This work proposes a novel system for off-line handwritten signature verification. A new descriptor founded on a quad-tree structure of the Histogram Of Templates (HOT) is introduced. For the verification step, we propose a robust implementation of the Artificial Immune Recognition System (AIRS). This classifier is inspired from the natural immune system, which generates antibodies to protect the human body against antigens. The AIRS training develops new memory cells that are subsequently used to recognize data through a k Nearest Neighbor (kNN) classification. Presently, to get a robust verification, the kNN classification is substituted by a Support Vector (SV) decision, yielding the AIRSV classifier. Experiments are performed on three datasets, namely, MCYT-75, GPDS-300 and GPDS-4000. AIRSV performance is assessed comparatively to both conventional AIRS as well as SVM. Obtained results demonstrated that AIRSV is more effective than classical AIRS. Moreover, the proposed signature verification system gives similar and sometimes better performance than SVM as well as the state-of-the-art methods.