In the vision of Industry 5.0, collaborative robots (or cobots) play a central supporting role in various industries, especially manufacturing. Close interaction with cobots requires special attention to user experience to fully exploit the benefits of this paradigm. Consequently, understanding the impact of a cobot’s physical size on user experience becomes critical to optimizing human–robot collaboration (HRC). This research aims to investigate the relationship between cobot size (UR3e – small cobot vs. UR10e – large cobot) and user experience in HRC contexts, in conjunction with other factors (i.e., cobot movement speed and product assembly complexity). Through a series of controlled experiments involving 32 participants, user experience data were obtained by collecting physiological measures (i.e., electro-dermal activity, heart activity, eye-tracking metrics) and subjective responses with questionnaires (i.e., perceived workload, interaction quality, and affective state). Results showed that the large cobot was generally perceived to be safer, more natural, efficient, fluid, and trustworthy. With the large cobot, there was a decrease in dominance; however, it was offset by the learning effect. Perceived workload was mainly influenced by product complexity. No clear difference in terms of mental strain emerged from the physiological data comparing the cobot sizes. In addition, the interaction term between cobot size and cobot movement speed never emerged as significant. The results of this research can offer practical insights to improve the effectiveness and acceptance of cobots during the implementation phase.