As cryo-electron microscopy (cryo-EM) has become a more commonly used method in structural biology, many structure models of biomolecules have been determined from cryo-EM density maps. However, our recent research revealed that many protein structure models determined with cryo-EM and deposited in the Protein Data Bank (PDB) have potential errors. Such errors include cases where modeled local conformation is not accurate and situations where misassignment of amino acid types. Thus, establishing a computational protocol that evaluates the model accuracy and corrects low-quality regions in the model is crucial to ensuring the quality of structure models deposited to the public database, PDB. Here, we present a new protocol (DAQ-refine) for evaluating a protein model built from a cryo-EM map and for applying local structure refinement in case the model has potential errors. In the DAQ-refine protocol, model evaluation is performed with a deep learning-based model-local map assessment score, DAQ score, which we developed recently. Then, the subsequent local refinement is performed by a modified procedure of AlphaFold2. In the local refinement step, we used a trimmed template and trimmed multiple sequence alignment as input of AlphaFold2 to control which structure regions to refine while leaving other more confident regions in the model intact. DAQ-refine showed overall the highest accuracy in structure refinement when compared with four other methods on protein structures in PDB that were built from cryo-EM maps determined at a 3-4.5 Å resolution. DAQ score was also able to identify the most accurate model from about 20 alternative models generated for each target protein. The results demonstrated that our protocol, DAQ-refine, consistently improves low-quality regions of initial models.