Job-Shop Scheduling Problem (JSP) with worker flexibility has gained significant attention in recent years, offering cost reduction and improved efficiency by limiting the number of workers. However, consideration of ergonomic factors is generally ignored for such systems. The ergonomic consideration is important to prevent stress, fatigue, and lack of focus among the workers, particularly when recurring manual tasks are involved. To prioritize productivity and workforce relief simultaneously, this article addresses the JSP with limited flexible workers considering ergonomic factors (JSPLFW-ER). The study focuses on worker workload and human energy expenditure as the key ergonomic factors. The average human energy expenditure rate is also considered to ensure that its value does not exceed the critical limit. An improved multi-objective discrete Jaya algorithm (IMDJA) is proposed to solve JSPLFW-ER by minimizing the makespan, maximum worker workload, and maximum human energy expenditure of the assigned operations. IMDJA incorporates a Jaya random partial updating mechanism in which a certain number of elements of the individual are randomly selected for Jaya updating. The number of elements to be selected for Jaya updating is evaluated using the Taguchi analysis. The performance of IMDJA is investigated with comprehensive experiments, using the 40 newly constructed instances and found competitive to solve JSPLFW-ER. Furthermore, a real-world industrial problem is presented with multiple experiments, which demonstrates the applicability of the JSPLFW-ER and effectiveness of the proposed IMDJA for solving the industrial problem.