A negative psychosocial work environment causes stress to the physicians of healthcare institutions, which affect their physical and mental health. This study aimed to evaluate the prevalence of psychosocial work factors and stress and their associations with the physical and mental health of hospital physicians in the Kaunas region of Lithuania. A cross-sectional study was performed. It was based on a questionnaire survey, which contained the Job Content Questionnaire (JCQ), three scales of Copenhagen Psychosocial Questionnaire (COPSOQ), and Medical Outcomes Study Short Form-36 (SF-36) health survey. The study was carried out in 2018. A total of 647 physicians completed the survey. Multivariate logistic regression models were performed by using the stepwise method. In the models potentially, confounding factors such as age and gender were controlled. In our study, the dependent variables were stress dimensions, and the independent variables were psychosocial work factors. The analysis showed that a quarter of surveyed physicians were classified as having low job skill discretion and decision-making authority, and the support of supervisors was weak. Approximately one-third of the respondents had low decision latitude, low co-worker support, and high job demands, and felt insecure at work. Job insecurity and gender were found to be the strongest independent variables for general and cognitive stress. The support of the supervisor was found as a significant factor in the case of somatic stress. Better evaluation of mental health was related to job skill discretion and co-workers' and supervisors' support, but it did not affect physical health. The confirmed associations suggest that looking at work organization factors, reducing exposure to stress, and increasing perception of the psychosocial environment can be linked to better subjective health evaluation.