Enzyme-linked immunosorbent assay (ELISA) is favored for its high throughput, low cost, and simplicity, but the limited sensitivity of the traditional colorimetric ELISA restricts its wider application. Therefore, developing improvement strategies to enhance its sensitivity is a current research hotspot. In this work, we developed a multivalent bifunctional nanobody (Nb28-C4-SBP) by fusing a nanobody gene (Nb28) against ochratoxin A (OTA) with a self-assembling peptide (C4) and a streptavidin-conjugated peptide (SBP), which can introduce streptavidin-conjugated polymerized horseradish peroxidase (SA-PolyHRP) as a probe into the detection system. Subsequently, a dual improved ELISA (DI-ELISA) was developed using the higher affinity multivalent bifunctional nanobody (Nb28-C4-SBP) and the more enzyme-loaded probe (SA-PolyHRP) for the ultrasensitive detection of OTA in coffee samples. The DI-ELISA demonstrated a detection limit of 1 pg/mL, which was about 20-fold lower than that of the Nb28-C4-SBP and SA-HRP based ELISA and 266-fold lower than that of the Nb28-SBP and SA-PolyHRP based ELISA, respectively. In summary, we developed an ultrasensitive, rapid, and simple immunoassay for detecting OTA in coffee samples. Additionally, the proposed improvement strategy based on multivalent nanobody and polymerized horseradish peroxide is straightforward to operate and holds great potential for enhancing the sensitivity in various immunoassays.