We consider the combined effect of readout errors and coherent errors, i.e., deterministic phase rotations, on the surface code. We use a recently developed numerical approach, via a mapping of the physical qubits to Majorana fermions. We show how to use this approach in the presence of readout errors, treated on the phenomenological level: perfect projective measurements with potentially incorrectly recorded outcomes, and multiple repeated measurement rounds. We find a threshold for this combination of errors, with an error rate close to the threshold of the corresponding incoherent error channel (random Pauli-Z and readout errors). The value of the threshold error rate, using the worst case fidelity as the measure of logical errors, is 2.6%. Below the threshold, scaling up the code leads to the rapid loss of coherence in the logical-level errors, but error rates that are greater than those of the corresponding incoherent error channel. We also vary the coherent and readout error rates independently, and find that the surface code is more sensitive to coherent errors than to readout errors. Our work extends the recent results on coherent errors with perfect readout to the experimentally more realistic situation where readout errors also occur.