The opening and closing of doors in buildings likely become challenging when earthquakes occur owing to the deformation of the door frames. Although this may delay evacuation, there are no anti-seismic standards for doors defined in the Japanese Building Standards Act, which was amended in 1981. Discussions on doors or door frames from a structural point of view are still rare, even though the behaviors of doors, door frames, and hinges under seismic excitation are critical for avoiding secondary disasters after earthquakes. In this study, a seismic response analysis of a building model with door frames was performed using the adaptively shifted integration (ASI)-Gauss code to find an approach to reduce the deformation of door frames. Numerical results confirmed that the severe damage to the non-structural wall caused by the interlayer deformation of the building allowed both door frames and doors to be severely deformed. The in-plane deformation angles exceeded the angles causing difficulty in opening the doors. A door acted as an element resisting the door frame from deforming on the upper floors, and the door function would be lost on the lower floors by deforming in the out-of-plane direction. Furthermore, the overall deformations of the door frames decreased significantly even on the lower floors, where the interlayer drift angle of the building was larger, by placing the door system in a setback position. This effect was greater when the setback length was longer.