Construction workers are at high risk of work-related musculoskeletal disorders (WMSDs) due to physically demanding manual-handling tasks in awkward postures. Although existing observational methods to identify ergonomic risks are inexpensive and easy to use, they are seldom used in construction sites because they are time-consuming, subject to observer bias, and require well-trained analysts. To address these drawbacks, this paper proposes a vision-based method to automatically classify workers' postures for ergonomic assessment. Specifically, it proposes a vision-based method that eliminates the need to collect extensive training-image datasets by employing classification algorithms to learn diverse postures from virtual images, and then identifies those postures in real-world images. The experimental tests showed about 89% classification accuracy in automatically classifying diverse postures on images, confirming the usefulness of virtual training images for posture classification. The proposed method has potential for automated ergonomic risk analysis, and could help to prevent WMSDs during diverse occupational tasks.