SARS-CoV-2 nucleocapsid (N) protein plays a key role in multiple stages of the viral life cycle such as viral replication and assembly. This protein is more conserved than the Spike protein of the virus and can induce both humoral and cell-mediated immune responses, thereby becoming a target for clinical diagnosis and vaccine development. However, the immunogenic characteristics of this protein during natural infection are still not completely understood. Patient-derived monoclonal antibodies (mAbs) against SARS-CoV-2 N protein were generated from memory B cells in the PBMCs using the antigen-specific B cell approach. For epitope mapping of the isolated hmAbs, a panel of series-truncated N proteins were used , which covered the N-terminal domain (NTD, aa 46-174 ) and C-terminal domain (CTD, aa 245-364 ), as well as the flanking regions of NTD and CTD. NTD- or CTD-specific Abs in the plasma from COVID-19 patients were also tested by ELISA method. Cross-binding of hmAbs or plasma Abs in COVID-19 patients to other human β-CoV N proteins was determined using the capture ELISA. We isolated five N-specific monoclonal antibodies (mAbs) from memory B cells in the peripheral blood of two convalescent COVID-19 patients. Epitope mapping revealed that three of the patient-derived mAbs (N3, N5 and N31) targeted the C-terminal domain (CTD), whereas two of the mAbs (N83 and 3B7) targeted the N-terminal domain (NTD) of SARS-CoV-2 N protein. All five patient-derived mAbs were cross-reactive to the N protein of SARS-CoV but showed little to no cross-reactivity to the N proteins of other human beta coronaviruses (β-CoVs). We also tested 52 plasma samples collected from convalescent COVID-19 patients for Abs against the N proteins of human β-CoVs and found that 78.8% of plasma samples showed detectable Abs against the N proteins of SARS-CoV-2 and SARS-CoV. No plasma sample had cross-reactive Abs to the N protein of MERS-CoV. Cross-reactive Abs to the N proteins of OC43 and HKU1 were detected in 36.5% (19/52) and 19.2% (10/52) of plasma samples, respectively. These results suggest that natural SARS-CoV-2 infection elicits cross-reactive Abs to the N protein of SARS-CoV and that the five patient-derived mAbs to SARS-CoV-2 N protein NTD and CTD cross-react with their counterparts of SARS-CoV, but not other human β-CoVs. Thus, these five patient-derived mAbs can potentially be used for developing the next generation of COVID-19 At-Home Test kits for rapid and specific screening of SARS-CoV-2 infection.