Value Trade-Offs in Matching Functions

Enabled by advances in information and communications technologies, open contests in online labor markets allow employers (individuals and companies of any size) to access a large pool of labor, thus creating an efficient alternative for sourcing labor globally with significantly lower costs compared to traditional markets. Given the importance of increasing the number of contestants in contests for labor to obtain better solutions, this paper examines the key factors that increase labor participation in open contests based on data from an online labor market. Employers benefit from more contestants since a larger pool of participants brings a more diverse set of ideas and potentially better quality solutions. We propose three categories of factors that affect participation in open contests: (1) contest design parameters (prize, duration, description length), (2) market environment factors (competition intensity and market price), and (3) project intrinsic characteristics (project complexity and project type). Our results stress the importance of market environment factors in shaping the number of participants in open contests in online labor markets beyond contest design parameters. Notably, we show that the proposed determinants are moderated by project type (ideation versus expertise), stressing the importance of distinguishing between types of projects that require different job skills. Implications for designing open contests in online labor markets are discussed.

Online labor markets (OLMs) such as upwork.com and freelancer.com are increasingly common channels for engaging with trained individuals for technical and analytical work. Yet, like much of the tech industry, OLMs suffer from under-representation of women. In this paper, we address the broad question of why this may be the case by focusing on project factors that deter women from participating. We study how task complexity and competition increase the risk profile of projects, and disproportionately affect the odds of women choosing to bid for projects, as well as their bid amount, relative to men. Furthermore, we examine the role of environmental ambiguity, induced by Covid-19, on the propensity of women to bid for such projects. We conducted two large, randomized experiments conducted on Amazon Mechanical Turk (AMT) before and during the pandemic. We vary task complexity and competition as experimental treatments within the experiments, while we treat the pandemic as an external shock that raises environmental ambiguity between the two experiments. We find that women are indeed deterred by project complexity in their bidding behavior but are more likely to bid when faced with competition pre-pandemic. However, during the pandemic, women forbear from competition, suggesting the presence of significant opportunity and transaction costs for women as they deal with online work during the pandemic. We contribute to the literature on STEM and gender diversity by establishing the specific factors affecting women’s participation and wages in OLMs and suggest several actionable managerial insights to make OLMs more inclusive and attractive to women.

Global online labor markets (OLMs) lower the barriers to entry and enable global competition for information technology (IT) services from providers around the world. Although the prior OLM literature predominantly found systematic advantages for IT service providers from developed countries because of their higher perceived quality, the reality is that most service providers in OLM are from developing countries. This phenomenon requires a robust analysis of how OLMs are evolving. In this study, we conduct a geo-economic analysis on IT service providers’ survival utilizing a unique longitudinal panel data set from an OLM, which comprises 40,874 IT service providers from different countries over a period of more than four years (2006 to 2010). Based on results from Survival models and a series of robustness checks, we were able to decipher how geo-economic factors (specifically the country development level) and reputation interact to determine service providers’ survival. Our findings provide a different perspective from the prior literature on OLM by showing a systematic advantage for IT service providers from developing countries in terms of survival, especially when providers from developing countries were able to signal their individual quality through reputation. We explain and discuss the mechanisms underlying these effects, and highlight implications for OLMs for IT services. The online appendix is available at https://doi.org/10.1287/isre.2017.0751 .

Freelancers in online labor markets often display many skills in their profiles to increase their chances of being hired. However, such behavior may lead to the skills they display straddling multiple categories, that is, “skill spanning.” In this paper, we extend the concept of category spanning into online labor markets in the form of skill spanning and empirically examine (1) how freelancers’ skill spanning affects employers’ hiring decisions for two different types of jobs (low- and high-skill jobs, respectively), and (2) how freelancers’ skill matching moderates the effects of skill spanning on employers’ hiring decisions. Based on a unique dataset of 12,428 high-skill jobs and 19,875 low-skill jobs on a leading online labor platform, we find that freelancers’ skill spanning has different impacts on employers’ hiring decisions for these two job types. Specifically, for high-skill jobs, freelancers’ skill spanning reduces their likelihood of winning contracts; however, for low-skill jobs, freelancers’ skill spanning and their probabilities of winning contracts demonstrate an inverse U-shape relationship. Furthermore, freelancers’ skill matching can moderate the negative effects of skill spanning for high-skill jobs but not for low-skill jobs. Our findings provide guidelines for different stakeholders in online labor markets, including freelancers and platform owners.

Online labor markets are Web-based platforms that enable buyers to identify and contract for information technology (IT) services with service providers using buyer-determined (BD) auctions. BD auctions in online labor markets either follow an open or a sealed bid format. We compare open and sealed bid auctions in online labor markets to identify which format is superior in terms of obtaining more bids and a higher buyer surplus. Our theoretical analysis suggests that the relative advantage of open versus sealed bid auctions hinges on the role of reducing service providers’ valuation uncertainty (difficulty in assessing the cost to execute a project) and competition uncertainty (difficulty in assessing the intensity of the competition from other service providers), which largely depend on the relative importance of the common value (versus the private value) component of the auctioned IT services, calling for an empirical investigation to compare open and sealed bid auctions. Based on a unique data set of 71,437 open bid auctions and 7,499 sealed bid auctions posted by 21,799 buyers at a leading online labor market, we find that, on average, although sealed bid auctions attract 18.4% more bids, open bid auctions offer buyers $10.87 higher surplus. Furthermore, open bid auctions are 55.3% more likely to result in a buyer’s selection of a certain service provider and 22.1% more likely to reach a contract (conditional on the buyer’s making a selection) with a provider, and they generate higher buyer satisfaction. In contrast to conventional wisdom that “the more bids the better” and industry practice of treating sealed bid auctions as a premium feature, our results suggest that the buyer surplus gained from the reduction in valuation uncertainty enabled by open bid auctions outweighs the buyer surplus gained from the higher competition uncertainty in sealed bid auctions, which renders open bid auctions a superior auction design in online labor markets.

Online labor markets, two-sided platforms that match buyers with freelancers for IT services, have become increasingly important for sourcing labor and creating jobs around the globe. However, matching buyers and freelancers is challenging, largely because of the difficulty in pricing idiosyncratic IT services, and buyers and freelancers face uncertainty over the price (termed value uncertainty) they should pay, or bid, for an IT service, respectively. We propose “bid price dispersion” as a key determinant of matching (the percentage of posted IT services that are actually contracted between buyers and freelancers), and we empirically examine the effect of bid price dispersion on the two key sequential stages of matching in online labor markets: (a) buyer indecision — whether a buyer offers a contract to any freelancer; and (b) freelancer regret — whether the freelancer accepts the contract offered by the buyer. Using panel data from a leading online labor market (Freelancer), our results show that bid price dispersion is associated with an increase in both buyer indecision and freelancer regret, thus hurting matching. The results are robust across several alternative model specifications and various measurements of bid price dispersion. We contribute to the literature on two-sided platforms by theorizing and empirically quantifying the negative effect of bid price dispersion on buyer-freelancer matching in online labor markets for IT services. We discuss the study’s practical implications for enhancing the design of online labor markets and the matching capability of two-sided platforms.

Online labor markets are web-based platforms that enable buyers to identify and contract for IT services with service providers using Buyer-Determined (BD) auctions. BD auctions in online labor markets either follow an open or a sealed bid format. We compare open and sealed bid auctions in online labor markets to identify which format is superior in terms of obtaining more bids and a higher buyer surplus. Our theoretical analysis suggests that the relative advantage of open versus sealed bid auctions hinges on the role of reducing service providers’ valuation uncertainty (difficulty in assessing the cost to execute a project) and competition uncertainty (difficulty in assessing the intensity of the competition from other service providers), which largely depends on the relative importance of the common value (versus the private value) component of the auctioned IT services, calling for an empirical investigation to compare open and sealed bid auctions. Based on a unique dataset of 71,437 open bid auctions and 7,499 sealed bid auctions posted by 21,799 buyers at a leading online labor market, we find that, an average, albeit sealed bid auctions attract 18.4% more bids, open bid auctions offer buyers $10.87 higher surplus. Furthermore, open bid auctions are 55.3% more likely to result in a buyer’s selection of a certain service provider, 22.1% more likely to reach a contract (conditional on the buyer’s making a selection) with a provider, and they generate higher buyer satisfaction. In contrast to conventional wisdom that “the more bids the better” and industry practice of treating sealed bid auctions as a premium feature, our results suggest that the buyer surplus gained from the reduction in valuation uncertainty enabled by open bid auctions outweighs the buyer surplus gained from the higher competition uncertainty in sealed bid auctions, which renders open bid auctions a superior auction design in online labor markets.

Online labor markets (OLM) lower the barriers of entry and enable global competition for IT service providers around the world. Although the prior OLM literature posits systematic advantages to IT service providers from developed countries, most providers in OLM are from developing countries. The jobs are flowing to the developing countries, while the employers remain in the developed countries. This emerging evidence requires fresh analysis to understand how OLMs are evolving. In this study, we conduct a geoeconomic analysis on IT service providers’ survival and wage growth, utilizing a unique longitudinal panel data set comprising 40,874 IT service providers from 150 different countries over a period of more than four years (2006 to 2010). Using Survival and Growth models, we uncover systematic advantages for IT service providers from developing countries in both survival and wage growth. We are also able to decipher trends in how these effects evolve over time as the marketplace matures. Contrary to prior literature on OLM reporting systematic advantages for IT service providers from developed countries in landing contracts, we found a systematic advantage for IT service providers from developing countries in terms of both survival and wage growth, especially when they were able to signal their individual quality. We explain and discuss the mechanisms underlying these effects, and highlight implications for online labor markets for IT services.

Despite some advantages over traditional (offline) labor markets – such as lower search costs, better matching and improved monitoring – online labor markets (OLMs) have not taken off as initially expected. In this paper, we study the factors that limit perceived project success on OLMs. Using psychological contract theory, we theorize how common OLM features including contracts with virtual monitoring, multi-freelancer projects, and simultaneous projects by a client trigger the perception of psychological contract breach among OLM participants and reduce perceived project success for both participants. We test these hypotheses using an extensive dataset with more than 143,000 transactions on the world’s largest freelancing platform, Upwork, and find that – contrary to predictions from agency theory – projects equipped with strict freelancer monitoring (hourly-pay contracts) and projects enabling peer comparison (multi-freelancer projects or multiple simultaneous projects), lead to lower perceived project success both from the freelancer’s and the client’s perspective. Our work implies that transactions on online labor markets should not be viewed solely as agency relations, and that some features that supposedly reduce agency costs and improve efficiency on OLMs come at the cost of triggering the perception of psychological contract breach.

Despite some advantages over traditional (offline) labor markets – such as lower search costs, better matching and improved monitoring – online labor markets (OLMs) have not taken off as initially expected. In this paper, we study the factors that limit perceived project success on OLMs. Using psychological contract theory, we theorize how common OLM features including contracts with virtual monitoring, multi-freelancer projects, and simultaneous projects by a client trigger the perception of psychological contract breach among OLM participants and reduce perceived project success for both participants. We test these hypotheses using an extensive dataset with more than 143,000 transactions on the world’s largest freelancing platform, Upwork, and find that – contrary to predictions from agency theory – projects equipped with strict freelancer monitoring (hourly-pay contracts) and projects enabling peer comparison (multi-freelancer projects or multiple simultaneous projects), lead to lower perceived project success both from the freelancer’s and the client’s perspective. Our work implies that transactions on online labor markets should not be viewed solely as agency relations, and that some features that supposedly reduce agency costs and improve efficiency on OLMs come at the cost of triggering the perception of psychological contract breach.

In online labor markets, experts sell their expertise to buyers. Despite the success and the perceived promise of online labor markets, they face a serious practical challenge: providing appropriate incentives for experts to participate and exert effort to accurately (successfully) complete tasks. Personal rating schemes have been proposed to address this challenge: they provide differentiated reward/punishment to experts in order to incentivize them to cooperate (i.e. to their best to complete tasks). However, when the transactions in a market are subject to errors, the experts are wrongly punished frequently whenever personal rating schemes are deployed. This not only reduces the experts' incentives to cooperate, but also it harms the market performance such as the obtained social welfare or revenue. To mitigate the problem of wrong punishments, we develop a novel game-theoretic formalism based on collective ratings. We formalize an online labor market as a two-sided trading platform where buyers and experts interact repeatedly. The market designer's problem is to create a market policy that maximizes the market's revenue subject to the constraints imposed by the characteristics of the market and the incentives of the participants. We propose to organize such markets by dividing experts into groups for which a collective rating is created and maintained based on the buyers' aggregated feedback. We analyze how the group size and the adopted rating scheme affect the market's revenue and the social welfare of the participants in the market, and determine the optimal design of the market policy. We show that collective ratings are surprisingly more effective and more robust than personal rating for a wide variety of online labor markets.

This study examines the effects of reputation in the nascent but rapidly growing online labor markets. In these markets, contract winners (vendors) provide clients with customized products such as computer software, business plans, and artistic designs. The products are used primarily for business purposes and require time for production after project-specific contracts are awarded. These characteristics render it unclear whether online reputation will have similar effects as in online retailing, where finished and standardized products are sold for consumption. We analyze field transaction data from a major online labor market. The analyses using matched contract samples and vendor panels consistently show that despite the governing power provided by contracts as well as the litigation and arbitration options, vendors’ online reputation can still be influential on clients. Vendors who have no reputation ratings are less likely to be chosen, and those with higher ratings are more likely to win subsequent bids. Importantly, however, such influences depend on the contract form that is used for a particular transaction—they are significant in output-based contracts but nonsignificant in input-based contracts. Besides extending the research on online reputation to the markets of customized production, this study shows contract form as an important boundary condition for the effectiveness of reputational information. It also provides direct managerial implications for electronic commerce in general and online labor markets in particular. This paper was accepted by Pradeep Chintagunta, marketing.

This article reports on an investigation of the role of lock-in exploitation and the impact of reputation portability on workers’ switching behaviors in online labor markets. Online platforms using reputation mechanisms typically prevent users from transferring their ratings to other platforms, inducing lock-in effects and high switching costs and leaving users vulnerable to platform exploitation. With a theoretical model, in which workers in online labor markets are locked-in by their reputational data, we test the effects using an online lab-in-the-field decision experiment. In addition to comparing a policy regime with and without reputation portability, we vary lock-in exploitation using platform fees to consider how switching behavior might differ according to monetary motives and fairness preferences. Theoretically, this study reveals how reputational investments can produce switching costs that platforms can exploit. Experimentally, the results suggest that reputation portability mitigates lock-in effects, making users less susceptible to lock-in exploitation. The data further show that switching is driven primarily by monetary motives, but perceiving the fee as unfair also has a significant role.

Online labor markets and the humans that power them serve a critical role in the advancement of artificial intelligence and supervised machine learning via the creation of useful training datasets. The use of human effort in online labor markets is not enough, however, as a key factor is understanding the possible interventions that market operators can leverage to incentivize human effort among their labor force. We propose that platforms could implement mechanisms such as rewards or punishments at individual or group levels to incentivize real-effort and output. We apply our interventions using a collaborative image tagging experiment—a folksonomy—and the results provide interesting insights and nonobvious consequences. On average, interventions applied at the group level outperformed interventions applied at the individual level. Punishing the group provided the most controversial incentive strategy and provided a nonobvious significant improvement in effort. Rewarding or sanctioning an individual had similar effects on average, with both treatments leading to significant increases in effort post-intervention. In contrast to predictions, sanctioning appears to have significantly motivated those that were punished. Overall, the interventions applied in our real-effort collaborative image tagging experiment had a significant impact on behavior, which provides guidance for online labor market operators and the use of incentives in the creation of labeled machine learning training datasets.

In the past decade, IT has facilitated the shift from permanent employment to need-based outsourcing and from local labor market to global online labor markets. While prior studies have examined how global frictions affect employers’ hiring decisions on online labor markets, we have limited understanding of the inter-dependence between workers and employers and the economic impact of IT-enabled globalization on matching outcomes such as the number of matched projects, freelancer wages, and project values generated from matching. This study is an attempt to fill in the gap by examining the dual roles of IT-enabled globalization, i.e., (1) in determining the formation of matches between employers and freelancers, and (2) in affecting market outcomes. From a market perspective, we take into account two-sided decision making, competition on each side, complementarities between employer and freelancer attributes, and endogenous money transfers between employers and freelancers.In our empirical analysis, we estimate a structural two-sided matching model of the online labor market from a revealed preference perspective. The estimation is based on a dataset from a major freelancing website that connects freelancers and employers from more than 200 countries. We then conduct counterfactual analysis to quantify the economic impact of IT-enabled globalization in online labor market by comparing the current scenario with a counterfactual scenario where employers can only match with freelancers from the same country. The results from our estimation suggest that employers tend to match with freelancers from the same country, and that employers from developed countries tend to match with freelancers from developing countries. The results from the counterfactual analysis suggest that IT-enabled globalization leads to more employers and freelancers with successful matches, lower average wage among matched freelancers, and higher total project values generated on the market.