Companies face the challenges of attracting solvers and motivating them to dedicate their time and effort to develop solutions in crowdsourcing contests. Previous research emphasizes the importance of crowdsourcing contest design for fostering solvers' engagement. However, even though contests are designed as a combination of various design elements, such as seeker's identity disclosure, seeker's status, contest duration and monetary prize, prior studies primarily focused on independent effects of these elements, ignoring their interdependency. Building our study on the social exchange theory, we adopt the neo-configurational perspective to explore effective designs of crowdsourcing contests and contribute to our understanding of how contest design elements interact to jointly stimulate solvers' engagement. By the means of the fuzzy-set Qualitative Comparative Analysis of 277 contests, we show that the combination of long contest duration and high monetary prize plays a significant role in attracting solvers to join crowdsourcing contests, while these elements need to be additionally reinforced by the seeker's identity disclosure and its high status to motivate solvers to actually commit to problem solving. In this way, our study provides unique insights about the differences between contest designs that only initially attract solvers and designs that actually motivate them to create value for seeker companies. It fills the gap where previous research failed to embrace the configurational character of contest design, while offering unique guidance to managers who struggle to attract solvers to join and contribute to crowdsourcing initiatives.