Spin foam models, an approach to defining the dynamics of loop quantum gravity, make use of the Plebanski formulation of gravity, in which gravity is recovered from a topological field theory via certain constraints called simplicity constraints. However, the simplicity constraints in their usual form select more than just one gravitational sector as well as a degenerate sector. This was shown, in previous work, to be the reason for the "extra" terms appearing in the semiclassical limit of the Euclidean EPRL amplitude. In this previous work, a way to eliminate the extra sectors, and hence terms, was developed, leading to the what was called the Euclidean proper vertex amplitude. In the present work, these results are extended to the Lorentzian signature, establishing what is called the Lorentzian proper vertex amplitude. This extension is non-trivial and involves a number of new elements since, for Lorentzian bivectors, the split into self-dual and anti-self-dual parts, on which the Euclidean derivation was based, is no longer available. In fact, the classical parts of the present derivation provide not only an extension to the Lorentzian case, but also, with minor modifications, provide a new, more four dimensionally covariant derivation for the Euclidean case. The new elements in the quantum part of the derivation are due to the different structure of unitary representations of the Lorentz group.