The main aim of this paper is to provide characterizations of Birkhoff–James orthogonality (BJ-orthogonality in short) in a number of families of Banach spaces in terms of the elements of significant subsets of the unit ball of their dual spaces, which makes the characterizations more applicable. The tool to do so is a fine study of the abstract numerical range and its relation with the BJ-orthogonality. Among other results, we provide a characterization of BJ-orthogonality for spaces of vector-valued bounded functions in terms of the domain set and the dual of the target space, which is applied to get results for spaces of vector-valued continuous functions, uniform algebras, Lipschitz maps, injective tensor products, bounded linear operators with respect to the operator norm and to the numerical radius, multilinear maps, and polynomials. Next, we study possible extensions of the well-known Bhatia–Šemrl theorem on BJ-orthogonality of matrices, showing results in spaces of vector-valued continuous functions, compact linear operators on reflexive spaces, and finite Blaschke products. Finally, we find applications of our results to the study of spear vectors and spear operators. We show that no smooth point of a Banach space can be BJ-orthogonal to a spear vector of Z. As a consequence, if X is a Banach space containing strongly exposed points and Y is a smooth Banach space with dimension at least two, then there are no spear operators from X to Y. Particularizing this result to the identity operator, we show that a smooth Banach space containing strongly exposed points has numerical index strictly smaller than one. These latter results partially solve some open problems.