Most antibodies used in immunohistochemistry (IHC) have been developed by animal immunization. We wanted to explore naive antibody repertoires displayed on filamentous phages as a source of full-length antibodies for IHC on Formalin-Fixed and Paraffin-Embedded (FFPE) tissues. We used two isogenic mouse fibroblast cell lines that express or not human HER2 to generate positive and negative FFPE pseudo-tissue respectively. Using these pseudo-tissues and previously described approaches based on differential panning, we isolated very efficient antibody clones, but not against HER2. To optimize HER2 targeting and tissue specificity, we first performed 3–4 rounds of in vitro panning using recombinant HER2 extracellular domain (ECD) to enrich the phage library in HER2 binders, followed by one panning round using the two FFPE pseudo-tissues to retain binders for IHC conditions. We then analyzed the bound phages using next-generation sequencing to identify antibody sequences specifically associated with the HER2-positive pseudo-tissue. Using this approach, the top-ranked clone identified by sequencing was specific to the HER2-positive pseudo-tissue and showed a staining pattern similar to that of the antibody used for the clinical diagnosis of HER2-positive breast cancer. However, we could not optimize staining on other tissues, showing that specificity was restricted to the tissue used for selection and screening. Therefore, future optimized protocols must consider tissue diversity early during the selection by panning using a wide collection of tissue types.