Large-scale Empirical Study Research Articles

Towards Better Comprehension of Breaking Changes in the NPM Ecosystem

Code evolution is prevalent in software ecosystems, which can provide many benefits, such as new features, bug fixes, security patches, etc., while still introducing breaking changes that make downstream projects fail to work. Breaking changes cause a lot of effort to both downstream and upstream developers: downstream developers need to adapt to breaking changes and upstream developers are responsible for identifying and documenting them. In the NPM ecosystem, characterized by frequent code changes and a high tolerance for making breaking changes, the effort is larger. For better comprehension of breaking changes in the NPM ecosystem and to enhance breaking change detection tools, we conduct a large-scale empirical study to investigate breaking changes in the NPM ecosystem. We construct a dataset of explicitly documented breaking changes from 381 popular NPM projects. We find that 95.4% of the detected breaking changes can be covered by developers’ documentation, and 19% of the breaking changes cannot be detected by regression testing. Then in the process of investigating source code of our collected breaking changes, we yield a taxonomy of JavaScript and TypeScript-specific syntactic breaking changes and a taxonomy of major types of behavioral breaking changes. Additionally, we investigate the reasons why developers make breaking changes in NPM and find three major reasons, i.e., to reduce code redundancy, to improve identifier names, and to improve API design, and each category contains several sub-items. We provide actionable implications for future research, e.g., automatic naming and renaming techniques should be applied in JavaScript projects to improve identifier names, future research can try to detect more types of behavioral breaking changes. By presenting the implications, we also discuss the weakness of automatic renaming and breaking change detection approaches, such as the lack of support for public identifiers and various types of breaking changes.

Association between dietary fiber intake and chronic kidney disease in adults with and without hypertension in the United States: a cross-sectional study of NHANES 2009–2020

While previous research has highlighted the potential advantages of increasing dietary fiber intake (DFI) for managing hypertension and chronic kidney disease (CKD), there is a gap in large-scale empirical studies examining the relationship between DFI and CKD among hypertensive and nonhypertensive cohorts independently. This study involved 22,871 participants sourced from the NHANES database spanning 2009 to 2020, who were divided into hypertensive (n = 9,861) and nonhypertensive (n = 13,010) groups. The analysis revealed a significant inverse correlation between DFI and CKD prevalence across the sample after adjusting for various covariates (OR = 0.98, 95% CI: 0.97–0.99, p = 0.001). Within the subset of hypertensive individuals, this inverse association mirrors the findings of the overall sample, indicating that a higher DFI was associated with a reduced occurrence of CKD (OR = 0.97, 95% CI: 0.96–0.99, p < 0.001). However, this correlation was not detected in the nonhypertensive group (OR = 0.99, 95% CI: 0.98–1.01, p = 0.285). The RCS analysis further confirmed a pronounced nonlinear inverse relationship between DFI and CKD prevalence in both the entire cohort and the hypertensive group but not in the nonhypertensive group. Further scrutiny of the hypertensive group revealed that individuals with a higher DFI had 33% lower odds of CKD progression for the moderate risk level and 36% lower odds for the high to very high risk level. Subgroup analyses confirmed the consistency of these relationships across various demographics. In summary, this investigation revealed a significant inverse relationship between DFI and CKD prevalence in US adults with hypertension, a relationship not observed in nonhypertensive individuals.

Word-Final /s/-/z/ Omission in Vietnamese English

Southeast Asian learners of English, including those from Vietnam, frequently omit word-final consonants in their English speech. Previous work on Vietnamese learners of English is limited, and errors are typically usually attributed to first-language transfer effects. No large-scale empirical study on Vietnamese learners has been carried out to aid the development of an evidence-based pedagogy. This study uses authentic spoken data to compare lexical and morphological word-final /s/ and /z/ in the speech of sixteen Vietnamese adult learners of English. We discuss the relative impact of frequency of use, whether the instance of a target /s/ or /z/ is in a root or bound morpheme, and whether the preceding phoneme is a consonant or vowel. An overall omission rate of 28.4% of expected instances was found. Morphological {-s} when it is preceded by a consonant has the highest error rate (50.7%). A multilevel binary logistic regression was performed to ascertain the relative effects. Morphological words containing /s/ or /z/ were significantly more likely to be pronounced with the /s/ or /z/ absent than lexical words containing a /s/ or /z/, as were those in clusters compared to those with a preceding vowel. The results indicate that phonological effects and morphological effects are stacked and not multiplicative and that the observed omission rates are not solely attributable to L1 transfer effects. Frequency of use is also highly correlated with accuracy.

Analysis of Profiles of Supporters of Conspiracy Narratives about Vaccination Against COVID-19 on a Social Network

The article analyzes the socio-demographic characteristics and value-worldview self-positioning of the distributors of conspiracy narratives (“conspiracy theories”) about vaccination against COVID-19 on the Russian social network “VK”. The research material consists of 33,674 profiles of VK users who published at least two messages during the pandemic that explicate the stated narrative. The collection and analysis of the material was carried out using big data analysis technologies. The purpose of the study is to determine whether it is possible to identify typical socio-demographic and ideological traits of anti-vaccer conspiracy theorists, which to a certain extent explain their penchant for conspiracy thinking. The results obtained refute the marginal dominance of conspiracy theorists in a pandemic situation. Versions popular in the scientific literature about the predominance of insufficiently educated people among “conspiracy theorists”, or the elderly as more gullible, or women as more suggestible, or residents of villages and small provincial towns as more conservative, etc. are not confirmed by the large-scale empirical studies of large user data. The results of the study may be useful for developing countermeasures and strategies to address public mistrust of institutions and official trends in public health, in particular vaccination policies. The traditional focus on information and educational measures (explanations about the dangers of the disease and the benefits of the vaccine) seems to be ineffective in the context of the emergence of value systems, cognitive distortions, general socio-political tension and other reasons that are still awaiting serious study.

A toolbox for surfacing health equity harms and biases in large language models.

Large language models (LLMs) hold promise to serve complex health information needs but also have the potential to introduce harm and exacerbate health disparities. Reliably evaluating equity-related model failures is a critical step toward developing systems that promote health equity. We present resources and methodologies for surfacing biases with potential to precipitate equity-related harms in long-form, LLM-generated answers to medical questions and conduct a large-scale empirical case study with the Med-PaLM 2 LLM. Our contributions include a multifactorial framework for human assessment of LLM-generated answers for biases and EquityMedQA, a collection of seven datasets enriched for adversarial queries. Both our human assessment framework and our dataset design process are grounded in an iterative participatory approach and review of Med-PaLM 2 answers. Through our empirical study, we find that our approach surfaces biases that may be missed by narrower evaluation approaches. Our experience underscores the importance of using diverse assessment methodologies and involving raters of varying backgrounds and expertise. While our approach is not sufficient to holistically assess whether the deployment of an artificial intelligence (AI) system promotes equitable health outcomes, we hope that it can be leveraged and built upon toward a shared goal of LLMs that promote accessible and equitable healthcare.

How can firms get benefits from the innovation ecosystem? Empirical evidence from Pakistan

PurposeThis research intends to analyze the innovation ecosystem factors that play a vital role in firm performance. As a result, large-scale empirical studies on the innovation ecosystem are rare, and fewer efforts have been made to determine if and how different factors affect the ecosystem models of firms. There has yet to be a substantial empirical study on the innovation ecosystem.Design/methodology/approachData were acquired from Pakistani IT companies. The results show that factors of the innovation ecosystem significantly contribute to business performance. The essential assumption is that resource endowment, organizational culture, knowledge and competence, and technology capability are allied to the innovation ecosystem.FindingsThe findings are crucial from a managerial view because firms must focus on changing their innovation ecosystem factors model to achieve greater performance. Radical changes in the firms will only be worthwhile if they value their resource endowments. To attain superior firm performance via influential factors of the innovation ecosystem, IT administrators need to build organizational cultural capacities to adapt to changes brought on by digitization quickly and effectively. However, this must be supplemented by improving organizational knowledge, competencies and technological capabilities to enable organizations to modify their ecosystems.Originality/valueEventually, firms can better respond to changes in their settings if they combine these variables by implementing an effective innovation ecosystem model, which leads to greater sector and superior financial performance.

THE USE OF SCHOLARSHIP BY THE EUROPEAN COURT OF HUMAN RIGHTS

AbstractThe relationship between scholarship and adjudication has attracted considerable attention in recent years, especially in those areas where significant academic expertise has been developed and academic scrutiny of decisions is common. Yet the role of scholars and scholarship in the context of the adjudicatory practices of the European Court of Human Rights (ECtHR) has remained palpably under-investigated. This article begins to fill this gap in the literature by carrying out the first large-scale empirical study of the use of scholarship by the ECtHR. The authors rely on a purpose-built dataset comprising all the citations made by the Grand Chamber of the Court in judgments and separate opinions appended to it. The study finds that the Court's majority uses scholarship for the purposes of reviewing facts and interpreting international and domestic law but does so rarely. The majority of the ECtHR does not use scholarship to interpret the European Convention on Human Rights or for persuasive purposes, unlike the individual Judges in their separate opinions. Indeed, individual Judges refer to scholarship more often, for more varied and arguably different purposes. This use, however, is inconsistent in terms of both frequency and the types of sources referred to.

An empirical analysis of electric vehicle charging behavior based on real Danish residential charging data

In the last two years there has been a significant increase in the global adoption of electric vehicles (EVs). Particularly in Denmark, 40% of new vehicle registrations in 2023 were electric, marking a notable shift towards e-mobility. As a result, EV loading begins to constitute a large part of residential electricity demand. The concurrent increase in the interest for reduction in charging costs via demand response leads to a growing need for better understanding of charging behavior and its impact on consumption profiles. This works addresses the lack of up-to-date large-scale empirical studies on residential EV charging behavior. It presents a thorough analysis of all important charging characteristics based on a large set of 5534 Danish residential chargers between 2021 and 2023, providing unique insight in the effect of user-induced controlled charging. Our analysis shows the difference between actual charging profiles and those commonly used by distribution system operators in Denmark, and reveals the significant change over the span of one year with the doubling of peak demand per charger from 1.25 kW to 2.5 kW.

An empirical study of fault localization in Python programs

Despite its massive popularity as a programming language, especially in novel domains like data science programs, there is comparatively little research about fault localization that targets Python. Even though it is plausible that several findings about programming languages like C/C++ and Java—the most common choices for fault localization research—carry over to other languages, whether the dynamic nature of Python and how the language is used in practice affect the capabilities of classic fault localization approaches remain open questions to investigate. This paper is the first multi-family large-scale empirical study of fault localization on real-world Python programs and faults. Using Zou et al.’s recent large-scale empirical study of fault localization in Java (Zou et al. 2021) as the basis of our study, we investigated the effectiveness (i.e., localization accuracy), efficiency (i.e., runtime performance), and other features (e.g., different entity granularities) of seven well-known fault-localization techniques in four families (spectrum-based, mutation-based, predicate switching, and stack-trace based) on 135 faults from 13 open-source Python projects from the BugsInPy curated collection (Widyasari et al. 2020). The results replicate for Python several results known about Java, and shed light on whether Python’s peculiarities affect the capabilities of fault localization. The replication package that accompanies this paper includes detailed data about our experiments, as well as the tool FauxPy that we implemented to conduct the study.

Analyzing the Energy Usage of a Community and the Benefits of Energy Storage

Understanding the energy usage of a community is crucial for policymaking, energy planning, and achieving sustainable development. The advent of the smart grid has made it feasible to gather fine-grain energy usage data at large-scales, providing us with new opportunities to understand demand patterns at different spatial and temporal scales. In this paper, we conduct a large-scale empirical study of energy usage of 14,849 residential and commercial energy consumers from a small city in the United States. We conduct a wide ranging analysis of energy usage at multiple granularities—citywide, transformer-level, and individual home levels. In doing so, we demonstrate how city-wide smart meter datasets can answer a variety of questions on energy consumption, such as the impact of weather on energy usage. For example, we show that extreme weather events significantly increase energy usage, e.g., by 36% and 11.5% on hot summer and cold winter days, respectively. As another example, we show 19.2% of transformers in the grid get overloaded during peak load periods. Finally, we evaluate the impact of incorporating varying amounts of energy storage within the distribution grid and the impact such deployments will have on the peak demand patterns seen by the grid as well as the ability to reduce overloads seen by distribution transformers during peak periods.

An initial exploration of semi-automated tutoring

In this paper, we begin our process of incorporating an AI bot in an online chat tutoring setting as a support for the tutor. We explore how an AI bot could give suggestions for tutor messages, although the human tutor will control how to communicate with the student. Tutoring, an important dimension of networked learning, has long been seen as a beneficial approach to students’ learning. An AI bot has the potential to aid tutors in the tutoring process and contribute to the scalability. The present pilot study was conducted in the tutoring setting of the Math Coach program. In the program, teacher students aid students from upper primary school to upper secondary school in mathematics through an online text-based chat system. Llama2 was used as a large language model (LLM), fine-tuned for Swedish comprehension utilizing the Math Coach system's chat logs. Four coaches, teacher students at a technical Swedish university and active in the Math Coach program, were invited to interact with the AI bot and participate in a group discussion. The coaches interacted individually with the AI bot while the chat conversation was displayed on a monitor so all participants could discuss the interaction while it took place. A semi-structured interview approach was taken and the participants were also encouraged to 'think aloud' about their experience. In the discussions, the coaches expressed surprise by the AI's social aspect. They perceived the AI bot as friendly with a positive attitude and were especially surprised by its ability to correctly place appropriate emojis. The coaches agreed that the AI was able to ask both appropriate and helpful questions and share some good guidance for how to proceed in the problem-solving process. However, they felt that the AI bot was not able to offer sufficient mathematical guidance, oftentimes the AI bot was confidently wrong. It also wrote too long messages, which humans would typically separate into several chat messages, and did not wait for a response but instead moved too quickly towards the solution. Moving forward we plan to address the effects of improved prompts on the AI bot and continue finetuning the LLM. We will continue to conduct pilot studies and eventually conduct more large-scale empirical studies.

Understanding the benefits and mechanisms of working from home as a sustainable urban strategy on air pollution: Empirical study in 19 cities in China

Working from home (WFH) arrangements are already acknowledged as a sustainable measure for alleviating urban congestion and air quality issues, but fewer large-scale empirical studies investigating the benefits and mechanisms of WFH on air pollution (AP). The exogenous shock of the epidemic provides unique conditions that enable us to conduct empirical studies on the positive impacts of working from home. Building upon this background, this paper designs a series of empirical models to investigate the benefits of WFH on AP and the mediating effect of intra-city travel intensity (ITI) concerning the WFH-AP relationship to depict the mechanism of the impact of WFH on AP. This paper collected panel data on ITI and AP from January 1 to March 15, 2020, in 19 cities in China for regression analyses. The findings indicate that WFH reduces ITI by 65.0 %, while an increase in ITI leads to higher AP emissions, particularly of NO2. WFH positively impacts urban air quality, resulting in a 26.3 % decrease in the Air Quality Index (AQI) and reductions of 17.2 %, 4.1 %, 32.7 %, 18.8 %, and 13.6 % for the five air pollution categories (CO, NO2, PM2.5, PM10, SO2) respectively. Intermediate analyses demonstrated that ITI partially mediated the decrease in AQI, PM2.5, and CO, while PM10 and NO2 showed complete mediation. However, ITI had a weak mediating effect on SO2 emissions. The contribution of this study is that it provides supporting evidence that working from home positively impacts urban air quality and commuting. Additionally, it offers urban administrators a fact-based approach to traffic management and environmental protection, pointing to important directions for future urban sustainability strategy development.

Societal and scientific impact of policy research: A large-scale empirical study of some explanatory factors using Altmetric and Overton

This study investigates how scientific research influences policymaking by analyzing citations of research articles in policy documents (policy impact) for nearly 125,000 articles across 434 public policy journals. We reveal distinct citation patterns between policymakers and other stakeholders like researchers, journalists, and the public. News and blog mentions, social media engagement, and open access publications (excluding fully open access) significantly increase the likelihood of a research article being cited in policy documents. Conversely, articles locked behind paywalls and those published under the full open access model (based on Altmetric data) have a lower chance of being policy-cited. Publication year and policy type show no significant influence. Our findings emphasize the crucial role of science communication channels like news media and social media in bridging the gap between research and policy. Interestingly, academic citations hold a weaker influence on policy citations compared to news mentions, suggesting a potential disconnect between how researchers reference research and how policymakers utilize it. This highlights the need for improved communication strategies to ensure research informs policy decisions more effectively. This study provides valuable insights for researchers, policymakers, and science communicators. Researchers can tailor their dissemination efforts to reach policymakers through media channels. Policymakers can leverage these findings to identify research with higher policy relevance. Science communicators can play a critical role in translating research for policymakers and fostering dialogue between the scientific and policymaking communities.

Mapping the landscape of university technology flows in China using patent assignment data

The commercialization of intellectual property has become a fundamental avenue for universities to improve regional competitiveness. However, large-scale empirical studies on patent transfers, reflecting technology and knowledge sharing, remain limited. This study used social network analysis to investigate university technology flows in China based on patent assignments from technological, organizational, and regional perspectives. The results firstly revealed clear stage characteristics in the number of university patent assignments with the improvement of the Chinese version of the Bayh-Dole Act. Secondly, popular technologies in university technology flows mainly concentrated on measurement, testing, digital transmission, and other areas in which enterprises lack international competitive advantage. Thirdly, central actors comprised prestigious universities focusing on science, engineering, and comprehensive disciplines, along with intellectual property-focused enterprises. Finally, university technology flows gradually delocalized, and varied regional patterns exist owing to a spatial mismatch between university knowledge supply and regional demand. These findings have several practical and policy implications for government and university management in terms of promoting emerging technologies and clarifying universities’ functions in regional and national innovation systems. This study contributes to the economic geography literature on regional innovation.

Enhancing the Online Learning Experience of Performing Arts Students in the Post-Pandemic Era: A Large-Scale Empirical Study in Hong Kong

Enhancing the Online Learning Experience of Performing Arts Students in the Post-Pandemic Era: A Large-Scale Empirical Study in Hong Kong

THE SIGNIFICANCE OF KNOWLEDGE MANAGEMENT PROCESSES FOR BUSINESS SUSTIANABILITY: THE ROLE OF SUSTAINABILITY-ORIENTED PROJECTS

The paper aims to investigate the correlation between knowledge management and business sustainability through sustainability-oriented projects. The role of projects in promoting sustainability is already recognized, with sustainability-oriented projects emerging as a pivotal mechanism for companies to actively contribute to sustainability goals. Theoretical arguments also suggest that the enhancement of knowledge management is crucial for the advancement of sustainable business. Still, assertions in both areas lack solid empirical verification. Simultaneously, there is a growing emphasis on exploring the connection between knowledge management and project management, as existing studies indicate that knowledge management enhances the efficiency and effectiveness of projects. However, the nexus between knowledge management, project management, and business sustainability remains largely unexplored. This study aims to address two questions: whether knowledge management has a positive effect on business sustainability, and whether it is mediated by sustainability-oriented projects. The research hypotheses are verified through a large-scale empirical study utilizing SEM.

Tourism and marine crises: The impact of Sargassum invasion on Caribbean small island developing sates

Since 2011 the Caribbean has been experiencing large amounts of Sargassum on its beaches causing disruptions to tourism. There are nonetheless no large-scale empirical studies across the regizon on the impact of Sargassum on tourism and local economies. In addressing this knowledge gap this study investigates the impact of Sargassum on sandy beaches on tourism and sandy beach economic activity across 30 Caribbean Small Island Developing States. Spectral reflectance data from the MODIS satellite mission and a Floating Algae Index were used to detect the presence of Sargassum on sandy beaches. Tourism activity was measured using monthly tourist arrivals data from the Caribbean Tourism Organization and near sandy beaches nightlights data from the United States Air Force Defense Meteorological Satellite Program-Operational Linescan System. A Fixed effects panel regression model which controls for island specific time invariant effects, monthly and yearly time effects, and island specific time trends and allow for cross-sectional dependence and serial correlation was adopted. The results demonstrate that the average incidence of Sargassum reduces tourist arrivals growth by 1.1 percentage points up to 8 months after an outbreak with the highest incidence event having a 9 percentage points reduction. The results from nightlight intensity show a fall in sandy beach economic activity up to 6 months after an outbreak. An average Sargassum event of 0.25 and 0.15 constitute a reduction in sandy beach nightlight intensity of about 1.6 and 0.9 percent respectively.

Deep Is Better? An Empirical Comparison of Information Retrieval and Deep Learning Approaches to Code Summarization

Code summarization aims to generate short functional descriptions for source code to facilitate code comprehension. While Information Retrieval (IR) approaches that leverage similar code snippets and corresponding summaries have led the early research, Deep Learning (DL) approaches that use neural models to capture statistical properties between code and summaries are now mainstream. Although some preliminary studies suggest that IR approaches are more effective in some cases, it is currently unclear how effective the existing approaches can be in general, where and why IR/DL approaches perform better, and whether the integration of IR and DL can achieve better performance. Consequently, there is an urgent need for a comprehensive study of the IR and DL code summarization approaches to provide guidance for future development in this area. This article presents the first large-scale empirical study of 18 IR, DL, and hybrid code summarization approaches on five benchmark datasets. We extensively compare different types of approaches using automatic metrics, we conduct quantitative and qualitative analyses of where and why IR and DL approaches perform better, respectively, and we also study hybrid approaches for assessing the effectiveness of integrating IR and DL. The study shows that the performance of IR approaches should not be underestimated, that while DL models perform better in predicting tokens from method signatures and capturing structural similarities in code, simple IR approaches tend to perform better in the presence of code with high similarity or long reference summaries, and that existing hybrid approaches do not perform as well as individual approaches in their respective areas of strength. Based on our findings, we discuss future research directions for better code summarization.

Sea-level rise and mental health among coastal communities: A quantitative survey and conditional process analysis

This is the first large-scale empirical study examining the impact of sea-level rise induced by climate change on mental health outcomes among coastal communities. The study focuses on Bangladesh, a country severely affected by salinity ingress, flood risks, and agricultural damage due to sea-level changes. Participants (n = 1,200) randomly selected from three coastal regions each having high, moderate, or low vulnerability to sea-level rise were surveyed during the pre-monsoon season in 2021. The cross-sectional survey included validated measures of psychological distress, depression, anxiety, stress, environmental stressors, resource loss, and demographics. The results indicated significantly higher levels of psychological distress, depression, anxiety, and stress in residents of high-vulnerability areas compared to moderate or low-vulnerability regions. Resource loss served as a mediating variable between environmental stressors and mental health outcomes. Furthermore, demographic analyses showed that older adults and women reported higher levels of psychological distress. These findings align with the Sendai Framework for Disaster Risk Reduction, highlighting urgent need for targeted mental health interventions and sustainable models of care in coastal areas increasingly threatened by sea-level rise.

Practical wisdom in the workplace: conceptualization, instrument development, and predictive power

PurposeThe purpose of this Real Impact Research Article is to empirically explore one of the most controversial and elusive concepts in knowledge management research – practical wisdom. It develops a 10-dimensional practical wisdom construct and tests it within the nomological network of counterproductive and productive knowledge behavior.Design/methodology/approachA survey instrument was created based on the extant literature. A model was developed and tested by means of Partial Least Squares with data obtained from 200 experienced employees recruited from CloudResearch Connect crowdsourcing platform.FindingsPractical wisdom is a multidimensional construct that may be operationalized and measured like other well-established knowledge management concepts. Practical wisdom guides employee counterproductive and productive knowledge behavior: it suppresses knowledge sabotage and knowledge hiding (whether general, evasive, playing dumb, rationalized or bullying) and promotes knowledge sharing. While all proposed dimensions contribute to employee practical wisdom, particularly salient are subject matter expertise, moral purpose in decision-making, self-reflection in the workplace and external reflection in the workplace. Unexpectedly, practical wisdom facilitates knowledge hoarding instead of reducing it.Practical implicationsManagers should realize that possessing practical wisdom is not limited to a group of select, high-level executives. Organizations may administer the practical wisdom questionnaire presented in this study to their workers to identify those who score the lowest, and invest in employee training programs that focus on the development of those attributes pertaining to the practical wisdom dimensions.Originality/valueThe concept of practical wisdom is a controversial topic that has both detractors and supporters. To the best of the author’s knowledge, this is the first large-scale empirical study of practical wisdom in the knowledge management domain.