The safety of structural systems depends on social, political, financial, organizational, behavioral, and other non-structural factors. These factors are related to use of the structure and its environment, malevolent and other unanticipated accidental loading, construction method and experience of construction crews, gross errors, monitoring to prevent abuse, etc. The above can broadly be classified as non-technical factors, involving significant epistemic uncertainty. In this manuscript, we formalize a procedure to handle non-technical factors and/or epistemic uncertainties in Robust, Reliability-based and Risk-based structural Design Optimization (R3DO). We propose that latent failure probabilities, subjective point-estimates reflecting non-technical factors and/or epistemic uncertainties, be added to the nominal member failure probabilities, calculated from objective aleatory uncertainty (technical factors). Latent failure probabilities are subjectively estimated by the analysis team, based on a holistic risk analysis addressing the technical and non-technical factors above. The relevance and impact of latent failure probabilities in R3DO have already been demonstrated elsewhere; these results are briefly recapped herein. We demonstrate that structural systems should be made redundant, not because of objective aleatory uncertainties in loads and material strengths, but to cope with the large impact of subjective epistemic uncertainties related to non-technical factors.