Best-worst Scaling Method Research Articles

Specificity ratings for English data.

A dataset of specificity ratings for English words is hereby presented, analyzed and discussed in relation with other collections of speaker-generated ratings, including concreteness. Both, specificity and concreteness are analyzed in their ability to explain decision latencies in lexical and semantic tasks, showing important individual contributions. Specificity ratings are collected through best-worst scaling method on the words included in the ANEW dataset (Bradley and Lang in Affective norms for English words (ANEW): instruction manual and affective ratings (Tech. Rep.). Technical report C-1, the center for research in psychophysiology, 1999), chosen for its compatibility with many other collections of rating resources, and for its comparability with Italian specificity data (Bolognesi and Caselli in Behav Res Methods 55(7):3531-3548, 2023), allowing for cross-linguistic comparisons. Results suggest that specificity plays an important role in word processing and the importance of taking specificity into consideration when investigating concreteness effects.

High monetary valuation of regulating forest ecosystem services in Japan: integrating the best-worst scaling and contingent valuation methods

ABSTRACT Effective forest management requires an understanding of socially prioritized forest ecosystem services and their integration into forest management design. Approximately 70% of Japan, our study area, is covered by forests. Therefore, prioritization-based resource allocation play a fundamental role in forest management in Japan. However, there is a lack of clarity regarding which forest ecosystem services the public evaluate the most. Thus, this study evaluates the social priority of 13 forest ecosystem services based on public preferences by employing the best-worst scaling (BWS) technique. Our study also assesses the value of such services by integrating a contingent valuation method with BWS. The survey was conducted using an online questionnaire for Japanese citizens enrolled in a survey company as representatives of the Japanese public in January 2017; 720 responses were analyzed. The BWS results revealed that the freshwater stock was the most valued forest ecosystem service, followed by carbon storage. However, citizens considered cultural ecosystem services to be of lower importance. The mean willingness to pay (WTP) for freshwater stock was approximately USD 23.77, while the mean WTP for inspiration for culture and art was lower at USD 1.13. Our findings indicate that the values of cultural and provisioning services are lower than Japanese taxes for forest management and suggest that forest management should prioritize regulating ecosystem services.

Exploring tourist preferences for local food: a Best-Worst Scaling analysis and market segmentation approach

PurposeLocal food impacts tourist satisfaction and drives the choice of a tourist destination. However, it is not clear which attributes of local food products drive tourists’ preferences. More specifically, little is known about potential segments in the tourist market. Acknowledging consumers’ divergent inclinations, we explore tourist preferences toward local food attributes through a market segmentation approach.Design/methodology/approachThis study uses the Best-Worst Scaling method to examine the preferences of 311 tourists for attributes of local food products that are known to affect their choices. We employ a Latent Class Analysis to identify market segments with distinct preferences.FindingsResults indicate that tourists prioritize taste, quality, authenticity, healthiness, connection to local culture and tradition and environmental friendliness over price, appearance, enhancement of local economy, availability and packaging. We further identify three segments: Sensory Seekers, Cultural Experiencers, and Price Conscious. The largest segment, Sensory Seekers, is driven by taste and quality attributes, while Cultural Experiencers prioritize connection to local culture and authenticity. The Price Conscious segment places a greater emphasis on price than the other segments. Cultural Experiencers demonstrate the highest willingness to pay for local food products.Practical implicationsThe positioning of local products should target Cultural Experiencers. Agri-firms and state agencies promoting a tourist destination internationally can also use the connection to local culture and authenticity in their marketing campaigns.Originality/valueOur research contributes to food marketing literature by showing which local food attributes drive tourist preferences. Moreover, we uncover unobservable heterogeneous preferences among tourists.

Evaluating Entity Importance in a Cross-National Context using Crowdsourcing and Best–Worst Scaling

Understanding the significance of entities in the context of a particular search topic necessitates a special focus on linguistic, cultural, and national aspects. This paper presents an innovative study on obtaining entity importance scores within a cross-national context. While setting up an extensive crowdsourcing task pool, we asked the crowd-workers to rank given entities within a particular general-domain search topic following the best-worst scaling method. Considering two different platforms (Amazon Mechanical Turk and Toloka AI), we focus on crowdworkers from the USA and Russia, speaking English and Russian, respectively. As a result of this, we reveal a strong impact of the national factor of crowdworkers on the entity importance annotations. By highlighting differences and commonalities in the importance scores across nations, our work provides advanced insights and a dataset for future research in the field of cross-national entity ranking. Our findings and insights can be applied beyond the general domain search topics, in particular, when working with recommendations (e.g., for eCommerce items) the consideration of the cross-national factor plays a vital role on the quality.

Measuring the importance of different barriers to opioid agonist treatment using best-worst scaling in an Australian setting

ObjectiveOpioid agonist treatment (OAT) is an effective treatment for opioid use disorder (OUD), however several client barriers to OAT are reported. Client importance of these barriers using economic preference elicitation measures have not been identified. This paper determines the most important OAT barriers using best-worst scaling (BWS) and compares the results of BWS to Likert scale. MethodsCross-sectional self-completed survey with 191 opioid dependent clients who attended Australian needle and syringe sites. Participants were presented 15 Likert scale barriers and 15 BWS barrier scenarios. The BWS data was presented using count analysis, multinomial logit and mixed logit models. The ranking of barrier items was completed using three BWS methods and one Likert scale method, with share preference results (BWS) or mean scores (Likert) used to rank the 15 barriers. ResultsThe most important client barriers were ‘enjoy using opioids’, ‘lack of support services’ and ‘hard to access’. The four ranking methods produced different barrier rankings for the most important barriers, but similar results for the least important barriers. ConclusionPolicies around OAT as a harm reduction approach, increased support services and increased availability of OAT services would be beneficial in improving OAT uptake. Comparing BWS and Likert methods produced different highest ranked barriers, indicating the method used to identify preferences has significant implications on the type of intervention prioritised.

Ethical Aspects of Physician Decision-Making for Deprescribing Among Older Adults With Dementia

Physicians endorse deprescribing of risky or unnecessary medications for older adults (aged ≥65 years) with dementia, but there is a lack of information on what influences decisions to deprescribe in this population. To understand how physicians make decisions to deprescribe for older adults with moderate dementia and ethical and pragmatic concerns influencing those decisions. A cross-sectional national mailed survey study of a random sample of 3000 primary care physicians from the American Medical Association Physician Masterfile who care for older adults was conducted from January 15 to December 31, 2021. The study randomized participants to consider 2 clinical scenarios in which a physician may decide to deprescribe a medication for older adults with moderate dementia: 1 in which the medication could cause an adverse drug event if continued and the other in which there is no evidence of benefit. Participants ranked 9 factors related to possible ethical and pragmatic concerns through best-worst scaling methods (from greatest barrier to smallest barrier to deprescribing). Conditional logit regression quantified the relative importance for each factor as a barrier to deprescribing. A total of 890 physicians (35.0%) returned surveys; 511 (57.4%) were male, and the mean (SD) years since graduation was 26.0 (11.7). Most physicians had a primary specialty in family practice (50.4% [449 of 890]) and internal medicine (43.5% [387 of 890]). A total of 689 surveys were sufficiently complete to analyze. In both clinical scenarios, the 2 greatest barriers to deprescribing were (1) the patient or family reporting symptomatic benefit from the medication (beneficence and autonomy) and (2) the medication having been prescribed by another physician (autonomy and nonmaleficence). The least influential factor was ease of paying for the medication (justice). Findings from this national survey study of primary care physicians suggests that understanding ethical aspects of physician decision-making can inform clinician education about medication management and deprescribing decisions for older adults with moderate dementia.

Deconstructing consumers’ low-carbon tourism promotion preference and its consequences: A heuristic-systematic model

In the context of low-carbon tour promotion, consumers lack the motivation to translate environmental awareness into practical action; moreover, it is unclear which form of promotion is the most effective for travel agencies. Thus, this study considers consumers’ preference for low-carbon promotion as an entry point and adopts the best-worst scaling method to analyze their preference characteristics. In addition, it introduces a heuristic-systematic model to analyze the influence of consumer preferences on travel agency image and low-carbon tourism behavior intention. The results show that consumer preferences are deconstructed into different types of tourism experiences according to heuristic and systematic paths, such that they create a positive effect on image evaluation and behavioral intentions. This study reveals the significance of consumer preference in low-carbon tour promotion scenarios, which helps break the promotion deadlock and promote tourism enterprises to optimize their service design.

Investigating Individuals' Preferences in Determining the Functions of Smartphone Apps for Fighting Pandemics: Best-Worst Scaling Survey Study.

Smartphone apps have been beneficial in controlling and preventing the COVID-19 pandemic. However, there is a gap in research surrounding the importance of smartphone app functions from a user's perspective. Although the insights and opinions of different stakeholders, such as policymakers and medical professionals, can influence the success of a public health policy, any strategy will face difficulty in achieving the expected effect if it is not based on a method that users can accept. This study aimed to assess the importance of a hypothetical smartphone app's functions for managing health during a pandemic based on the perspective of user preferences. A cross-sectional and web-based survey using the best-worst scaling (BWS) method was used to investigate the general population's preferences for important smartphone app functions. Participants were recruited from a professional surveying company's web-based surveying panel. The attributes of the BWS questionnaire were developed based on a robust process, including literature review, interviews, and expert discussion. A balanced incomplete block design was used to construct the choice task to ensure the effectiveness of the research design. Count analysis, conditional logit model analysis, and mixed logit analysis were used to estimate preference heterogeneity among respondents. The responses of 2153 participants were eligible for analysis. Nearly 55% (1192/2153) were female, and the mean age was 31.4 years. Most participants (1765/2153, 81.9%) had completed tertiary or higher education, and approximately 70% (1523/2153) were urban residents. The 3 most vital functions according to their selection were "surveillance and monitoring of infected cases," "quick self-screening," and "early detection of infected cases." The mixed logit regression model identified significant heterogeneity in preferences among respondents, and stratified analysis showed that some heterogeneities varied in respondents by demographics and COVID-19-related characteristics. Participants who preferred to use the app were more likely to assign a high weight to the preventive functions than those who did not prefer to use it. Conversely, participants who showed lower willingness to use the app tended to indicate a higher preference for supportive functions than those who preferred to use it. This study ranks the importance of smartphone app features that provide health care services during a pandemic based on the general population's preferences in China. It provides empirical evidence for decision-makers to develop eHealth policies and strategies that address future public health crises from a person-centered care perspective. Continued use of apps and smart investment in digital health can help improve health outcomes and reduce the burden of disease on individuals and communities.

An empirical comparison of conjoint and best-worst scaling case III methods

The Best-Worst Scaling (BWS) case III method, also called the BWS ‘multi-profile case.’ has been widely used to characterize survey respondent preferences for market goods. The BWS method is similar to conjoint analysis methods in that respondents select from a set of hypothetical item profiles with different attribute levels. Unlike conjoint methods, which allow respondents to select their best/most preferred profile, the BWS case III method asks respondents to select ‘best’ and ‘worst’ profiles in each choice set. This study compares consumer willingness to pay (WTP) estimates from conjoint and BWS case III survey formats. Data on consumer preferences for single-use eating-ware products made from biobased materials were collected. Results suggest that for the most preferred attribute levels, WTPs estimates are similar in magnitude and consistent for signs across methods. For least-preferred attributes, WTP estimates from the conjoint method are higher than those of the BWS method. However, the BWS WTP estimates have smaller confidence intervals.

Student-Centred Studio Environments: A Deep Dive into Architecture Students’ Needs

Architectural education has long placed a strong emphasis on the studio, which serves as a crucial place for teaching and learning. The architecture studio is a dynamic and lively place where students can engage in design discussions, exchange ideas, and receive feedback from instructors and peers. Despite recognising the vital role architecture studios play in shaping the learning experience of architecture students, limited research has explored which specific attributes of the studio environment are most important to students. This research aims to address this gap by determining the priorities of architecture students regarding key studio attributes. The research employs a quantitative approach using the Best-Worst Scaling (BWS) model and a questionnaire survey. A pilot study was conducted with undergraduate architecture students. The results reveal the ranking and relative importance of key studio attributes, organised into five categories: Physiological Facilities, Information and Communications Technology, Indoor Environment Quality, Territoriality, Furniture, and Reference. This research provides a valuable reference for designing student-centred studio environments and demonstrates the viability of using the BWS method to determine students’ priorities for studio attributes.

Farmers’ selection criteria for sweet potato varieties in Benin: An application of Best-Worst Scaling

Summary Integrating farmers’ preferences into the breeding and dissemination of new genotypes is a effective approach to enhance their successful adoption by farmers. In the case of sweet potato, a staple crop in many parts of West Africa, there is a need for more research on the selection criteria used by farmers when choosing which varieties to grow. This study aims to highlight farmers’ selection criteria for sweet potato varieties in the main production areas in Benin. A total of 480 farmers from the top three sweet potato production areas were surveyed. The relative importance of various traits for sweet potato farmers was evaluated using best-worst scaling methods. Latent class analysis was applied to find groups of farmers with similar preferences. Best-Worst Scaling analysis revealed that high root yield, root size, marketability, and early maturing were the most important variety selection criteria. Latent class analysis revealed three farmers’ groups referred to as ‘Yield potential’, ‘Market value’, and ‘Plant resilience’ classes. ‘Yield potential’ farmers were more likely to be from Atlantique and Alibori departments; they significantly committed more acreage to sweet potato production. The ‘Market value’ farmers highlighted the variety of root size and commercial value as the main selection criteria and consisted of farmers with primary education levels from the Ouémé department. ‘Plant resilience’ refers to a group of Alibori farmers who prioritize environmental issues and primarily grow sweet potatoes for self-consumption. Our findings shed light on farmers’ preferences and suggested that heterogeneity in sweet potato selection criteria was highly influenced by various socio-economic factors and location.

Factors Influencing Interest in Post-Secondary Agricultural International Travel Experiences

International travel experiences (ITE) are high-impact practices that allow post-secondary agricultural students the opportunity to engage with production processes and commodities in other countries. In addition to adding tremendous value to their technical knowledge base students are also able to learn about new cultures. Prior research has focused on factors influencing students’ decision to participate in ITE programs. This study builds on research using best-worst scaling to quantitatively analyze motivations or obstacles impacting students’ intention to participate in an ITE. This approach provides a preference share which can be directly compared to other motivations or obstacles in a manner that Likert scales cannot. Students majoring in agriculture at two comprehensive regional universities were surveyed on their motivations or hesitancy in participating in an ITE. Use of the best-worst scaling method reveals that the opportunity for personal growth and earning academic credit are the two primary reasons generating interest in an ITE while cost and concerns about not earning academic credit are the primary reasons for students’ hesitancy in participating. The results of this study can be used by university coordinators in utilization of their limited resources to improve ITE and overcome hesitancy of students.

The post-COVID-19 tourism recovery led by crisis-resistant tourists: Surf tourism preferences in the Canary Islands

The post-COVID-19 tourism recovery is most likely to be led by crisis-resistant tourists (Hajibaba, Gretzel, Leisch and Dolnicar, 2015). The surf tourism market is highly crisis resistant (Dolnicar and Fluker, 2003a) and has potential to expand during the pandemic (Mach and Ponting, 2021), especially with the advances in technologies for remote workers. In an attempt to answer Hritz and Franzidis' (2018) call for further research on this niche market in order to ascertain the significance of surf tourism in a particular community, we provide a baseline for surf tourism in terms of these niche preferences and satisfaction levels in Gran Canaria (Spain). The study is based on a discrete choice model that relies on preferences elicited through best-worst scaling methods. Our research demonstrates that the BWS scores can be used to develop a destination attribute quality index based on importance and satisfaction scales and provides important information to destination managers.

Civil society's perception of forest ecosystem services. A case study in the Western Alps.

Forest Ecosystem Services (FES) are widely recognised by the society nowadays. However, no study in the literature has analysed a ranking of FES after the pandemic. This paper investigated civil society’s perception and knowledge toward these services; in addition, the presence of attitudinal or behavioural patterns regarding individual’s preference, was assessed. A choice experiment was conducted using the Best-Worst Scaling (BWS) method on a sample of 479 individuals intercepted in the Argentera Valley, in the Western Italian Alps. Results, showed a strong interest in biodiversity, aesthetic landscape quality and psychophysical health and a lower interest in provisioning services. Based on the individual preferences, civil society was clustered into five groups for FES, named “Hedonistic,” “Individualist with cultural and health interests,” “Sensitive to regulatory and utilitarian functions,” “Climate change sensitive” and “Livelihood and hedonistic wellbeing.” In general, there was a growing appreciation by civil society for the intangible services offered by the forest, driven by modern lifestyles and an interest in learning more about the provided services. Based on these elements, we believe that similar research should be extended to other mountain contexts to validate the results or to find new insights, and that it is now necessary to study how to involve civil society in decision-making processes of forest planning and management at a local level.

Specificity ratings for Italian data.

Abstraction enables us to categorize experience, learn new information, and form judgments. Language arguably plays a crucial role in abstraction, providing us with words that vary in specificity (e.g., highly generic: tool vs. highly specific: muffler). Yet, human-generated ratings of word specificity are virtually absent. We hereby present a dataset of specificity ratings collected from Italian native speakers on a set of around 1K Italian words, using the Best-Worst Scaling method. Through a series of correlation studies, we show that human-generated specificity ratings have low correlation coefficients with specificity metrics extracted automatically from WordNet, suggesting that WordNet does not reflect the hierarchical relations of category inclusion present in the speakers' minds. Moreover, our ratings show low correlations with concreteness ratings, suggesting that the variables Specificity and Concreteness capture two separate aspects involved in abstraction and that specificity may need to be controlled for when investigating conceptual concreteness. Finally, through a series of regression studies we show that specificity explains a unique amount of variance in decision latencies (lexical decision task), suggesting that this variable has theoretical value. The results are discussed in relation to the concept and investigation of abstraction.

Radiation adverse outcome pathways (AOPs) are on the horizon: advancing radiation protection through an international Horizon-Style exercise

Purpose The Adverse Outcome Pathway (AOP) framework, a systematic tool that can link available mechanistic data with phenotypic outcomes of relevance to regulatory decision-making, is being explored in areas related to radiation risk assessment. To examine the challenges including the use of AOPs to support the radiation protection community, an international horizon-style exercise was initiated through the Organisation for Economic Co-operation and Development Nuclear Energy Agency High-Level Group on Low Dose Research Radiation/Chemical AOP Joint Topical Group. The objective of the HSE was to facilitate the collection of ideas from a range of experts, to short-list a set of priority research questions that could, if answered, improve the description of the radiation dose-response relationship for low dose/dose-rate exposures, as well as reduce uncertainties in estimating the risk of developing adverse health outcomes following such exposures. Materials and methods The HSE was guided by an international steering committee of radiation risk experts. In the first phase, research questions were solicited on areas that can be supported by the AOP framework, or challenges on the use of AOPs in radiation risk assessment. In the second phase, questions received were refined and sorted by the SC using a best-worst scaling method. During a virtual 3-day workshop, the list of questions was further narrowed. In the third phase, an international survey of the broader radiation protection community led to an orderly ranking of the top questions. Results Of the 271 questions solicited, 254 were accepted and categorized into 9 themes. These were further refined to the top 25 prioritized questions. Among these, the higher ranked questions will be considered as ‘important’ to drive future initiatives in the low dose radiation protection community. These included questions on the ability of AOPs to delineate responses across different levels of biological organization, and how AOPs could be applied to address research questions on radiation quality, doses or dose-rates, exposure time patterns and deliveries, and uncertainties in low dose/dose-rate effects. A better understanding of these concepts is required to support the use of the AOP framework in radiation risk assessment. Conclusion Through dissemination of these results and considerations on next steps, the JTG will address select priority questions to advance the development and use of AOPs in the radiation protection community. The major themes observed will be discussed in the context of their relevance to areas of research that support the system of radiation protection.

Best-worst scaling, an alternative method to assess perceptual sound qualities.

When designing sound evaluation experiments, researchers rely on listening test methods, such as rating scales (RS). This work aims to investigate the suitability of best-worst scaling (BWS) for the perceptual evaluation of sound qualities. To do so, 20 participants rated the "brightness" of a corpus of instrumental sounds (N = 100) with RS and BWS methods. The results show that BWS procedure is the fastest and that RS and BWS are equivalent in terms of performance. Interestingly, participants preferred BWS over RS. Therefore, BWS is an alternative method that reliably measures perceptual sound qualities and could be used in many-sounds paradigm.

A comparison of Best-Worst Scaling and Likert Scale methods on peer-to-peer accommodation attributes

Surveys based on Likert scales continue to dominate market research practice despite their limitations. Several researchers have suggested adopting different types of scales and a unique alternative for rating the importance level of several attributes is Best − Worst Scaling (BWS). The purpose of this study is to compare two scaling approaches, the Best-Worst Scale (BWS) and the Likert Scale to explore their advantages and disadvantages. This study tried to identify the relative importance of Peer-to-Peer (P2P) accommodation attributes using the aforementioned two scaling approaches. A comparison of the results found that the BWS approach helps to validate priorities from a customer perspective by achieving better discrimination among attributes, while the Likert scale approach is useful for comparing group differences such as gender differences.

The importance of local food products attributes in Brazil consumer's preferences

The consumption of local food products is a trend in several countries. Governments have developed strategies to stimulate the production and consumption of local food products, which in turn might empower the local economy. This study aimed to identify the importance of local food products attributes in Brazil. An online survey was conducted, and a dual-response Best-worst scaling method was applied to sample of 205 Brazilian consumers. Results of the Hierarchical Bayes analyses showed that the order from the most to the least important attributes were: (1) Healthiness, (2) Safety, (3) Animal welfare, (4) Environmental impact, (5) Support of the local economy, (6) Taste, (7) Price, (8) Availability, (9) Traceability, (10) Place of origin, (11) Convenience. Based on the results, managerial implications are presented.

Using stated preference methods to inform public health decision making

Croatia has recognised the importance of prevention programmes in the field of public health, although their effectiveness is not satisfactory due to the low population response, which has a negative impact on the rationalisation of public spending. One of the possible solutions is to consider the stated preferences of the target population. Therefore, the main objective of this study is to evaluate the use of stated preference methods in improving public health prevention programmes. For the purpose of the study, a questionnaire was designed using three different methods - Discrete Choice Experiment (DCE), Contingent Rating method (CR) and Best-Worst Scaling method (BWS). This study shows that the attributes of the Croatian cervical cancer screening programme are significantly associated with the respondents' utility level, which in turn is related to women's response. Since BWS, DCE, and CR measure the same construct - utility - we can say that convergent validity partially confirms the external validity of the methods. The author concludes that it is necessary to implement market principles, i.e. the demand-side analysis using stated preference methods, in the planning, implementation and re-evaluation of public health programmes.