Ranked List Research Articles

Online and Offline Evaluation in Search Clarification

The effectiveness of clarification question models in engaging users within search systems is currently constrained, casting doubt on their overall usefulness. To improve the performance of these models, it is crucial to employ assessment approaches that encompass both real-time feedback from users (online evaluation) and the characteristics of clarification questions evaluated through human assessment (offline evaluation). However, the relationship between online and offline evaluations has been debated in information retrieval. This study aims to investigate how this discordance holds in search clarification. We use user engagement as ground truth and employ several offline labels to investigate to what extent the offline ranked lists of clarification resemble the ideal ranked lists based on online user engagement. Contrary to the current understanding that offline evaluations fall short of supporting online evaluations, we indicate that when identifying the most engaging clarification questions from the user’s perspective, online and offline evaluations correspond with each other. We show that the query length does not influence the relationship between online and offline evaluations, and reducing uncertainty in online evaluation strengthens this relationship. We illustrate that an engaging clarification needs to excel from multiple perspectives, and SERP quality and characteristics of the clarification are equally important. We also investigate if human labels can enhance the performance of Large Language Models (LLMs) and Learning-to-Rank (LTR) models in identifying the most engaging clarification questions from the user’s perspective by incorporating offline evaluations as input features. Our results indicate that LTR models do not perform better than individual offline labels. However, GPT, an LLM, emerges as the standout performer, surpassing all LTR models and offline labels.

Risk Analysis of Level Crossing Element Failures in a Fuzzy Environment

The concept of risk analysis is especially important because it examines and analyses in a detailed manner the factors that affect the normal functioning of a system. In this paper, the level crossing is considered as one system, composed of several elements. The failures of those elements were analysed with the aim of showing which are the most frequent and most critical failures. A multi-methodological approach was used in the analysis. The failure modes and effects analysis (FMEA) method was used to determine risk factors, after that a multi-criteria model was created in a fuzzy environment, and as output, it gave a ranking list of critical failures in the system. Through the discussion of the results, a comparison of the basic model with two other similar ones was made, and the comparative results were analysed. The main aim of this paper is to present one of the possible ways to analyse the risk of the system of level crossings with the aim of improving traffic safety at the crossing.

Emergency Medicine Clerkship Grading Scheme, Grade, and Rank-List Distribution as Reported on Standardized Letters of Evaluation

Background: The Standardized Letter of Evaluation (SLOE) is a crucial component of the emergency medicine (EM) application process. Given the critical role of the SLOE, we attempted to better understand the grading scales used, as well as the distribution of grades and rank-list positions.Objectives: Our primary objective in this study was to determine the distribution of grading formats, grades given, and rank-list positions across EM clerkships using the SLOE. Methods: We performed a cross-sectional study of the grading formats, grades given, and ranking distributions as reported on the SLOE during the 2022–23 application cycle. We obtained data on SLOEs from EM residency programs accredited by the Accreditation Council for Graduate Medical Education by reviewing all applicants who applied to either of two EM residency programs in geographically different regions. Trained abstractors recorded the following data: number of students rotating in the prior year; grading format used; and grade and rank distribution among students. Results: We included 264 programs in our final analysis, after 13 programs met exclusion criteria. The majority of programs (72.2%) use an Honors/High Pass/Pass/Fail grading scheme. We determined the mean percent of each grade: Honors/A 27.6%; High Pass/B 31.1%; Pass/C 40.8%; Low Pass/D 0.2%; and Fail/F 0.3%. Finally, we determined the mean percent for each rank-list position: top 10% was 17.6%; top third 36.5%; mid third 34.1%; and low third 11.8%. Conclusion: We determined the grading schemes and grade and rank-list distributions for EM programs during the 2022–2023 academic year. Most programs used a Honors/High Pass/Pass/Fail grading scheme, with the majority of students receiving Honors or High Pass, while 0.3% failed their rotation. Both grades and rank list demonstrated evidence of a skewed distribution toward higher grades and rank-list position.

Adjective ‘Sudden’ in Phrases as a Means of Temporal Mitigation in English Fantasy Literature

This study explores the semantics of the temporal mitigation indicator, specifically the adjective ‘sudden,’ as a form of litotes in English-language fantasy prose from the 20th to 21st centuries. The aim of this research is to identify English phrases containing the adjective ‘sudden’ that serve as litotes in the representation of time, which are frequently employed in the specified texts. The methods utilized include semantic analysis, contextual analysis, and comparative analysis. The findings reveal a list of phrases actively used by various authors within the realm of 20th and 21st-century English-language fantasy prose that reflect a high degree of temporal mitigation within this genre. An analysis of selected fantasy texts indicates the presence of collocations featuring ‘sudden’ that are not documented in collocation dictionaries. A ranked list of these phrases, along with their respective authors, has been compiled. The examination of collocations expressing temporal mitigation shows that some are utilized by multiple authors, such as “the sudden darkness,” “sudden anxiety,” and “a sudden thought.” This suggests their stability, indicating that they could enhance existing collocation dictionaries of the English language. The practical significance of this study lies in its potential application in teaching courses on lexicology and lexicography of the English language at the university level.

Principal component analysis revisited: fast multitrait genetic evaluations with smooth convergence.

A cornerstone in breeding and population genetics is the genetic evaluation procedure, needed to make important decisions on population management. Multivariate mixed model analysis, in which many traits are considered jointly, utilizes genetic and environmental correlations between traits to improve the accuracy. However, the number of parameters in the multitrait model grows exponentially with the number of traits which reduces its scalability. Here, we suggest using principal component analysis to reduce the dimensions of the response variables, and then using the computed principal components as separate responses in the genetic evaluation analysis. As principal components are orthogonal to each other so that phenotypic covariance is abscent between principal components, a full multivariate analysis can be approximated by separate univariate analyses instead which should speed up computations considerably. We compared the approach to both traditional multivariate analysis and factor analytic approach in terms of computational requirement and rank lists according to predicted genetic merit on two forest tree datasets with 22 and 27 measured traits, respectively. Obtained rank lists of the top 50 individuals were in good agreement. Interestingly, the required computational time of the approach only took a few seconds without convergence issues, unlike the traditional approach which required considerably more time to run (7 and 10 h, respectively). The factor analytic approach took approximately 5-10 min. Our approach can easily handle missing data and can be used with all available linear mixed effect model softwares as it does not require any specific implementation. The approach can help to mitigate difficulties with multitrait genetic analysis in both breeding and wild populations.

The Rank Length Effect

Rank lists vary in the number of items ranked on the list (e.g., top 5 vs. top 20 movies on IMDb), that is, the rank length. Across ten studies, including both field and laboratory experiments, the authors examine the influence of rank length on evaluations, willingness to pay, and choice. They document a novel rank length effect: The same ranked items elicit more positive judgments when the rank length is longer (vs. shorter), although the differences in judgments between the ranked items are smaller. This effect is driven both by consumers’ tendency to narrowly focus on the rank list and by the manner in which they map the rank list onto their mental number line. The rank length effect extends to willingness to pay, and choice. The authors explore three different kinds of choice contexts, discuss implications, and offer suggestions for future research.

Multimorbidity in neurodegenerative diseases: a network analysis

ABSTRACT The socioeconomic costs of neurodegenerative diseases (NDs) are highly affected by comorbidities. This study aims to enhance our understanding of the prevalent complications of NDs through the lens of network analysis. A multimorbidity network (MN) was constructed based on a longitudinal EHR dataset of 93,647,498 diagnoses of 824,847 patients. The association between the conditions was measured by two metrics, i.e. Phi-correlation and Cosine Index (CI). Based on multiple network centrality measures, a fused ranking list of the prevalent multimorbidities was provided. Finally, class-level networks depicting the prevalence and strength of diseases in different classes were constructed. The general MN included 928 diseases and 337,253 associations. Considering a 99% confidence level, two networks of 575 relationships were constructed based on Phi-correlations (73 diseases) and CI (102 diseases). Five out of 19 ICD-9 categories did not appear in either of the networks. Also, ND’s immediate MNs for the top 50% of the significant associations included 42 relationships, whereas the Phi-correlation and CI networks included 36 and 34 diseases, respectively. Thirteen diseases were identified as the most notable multimorbidities based on various centrality measures. The analysis framework helps practitioners toward better resource allocations, more effective preventive screenings, and improved quality of life for ND patients and caregivers.

Machine learning model base on metabolomics and proteomics to predict cognitive impairment in Parkinson’s disease

There is an urgent need to identify predictive biomarkers of Parkinson’s disease (PD) with cognitive impairment (PDCI) in order to individualize patient management, ensure timely intervention, and improve prognosis. The aim of this study was to screen for these biomarkers by comparing the plasma proteome and metabolome of PD patients with or without cognitive impairment. Proteomics and metabolomics analyses were performed on a discover cohort. A machine learning model was used to identify candidate protein and metabolite biomarkers of PDCI, which were validated in an independent cohort. The predictive ability of these biomarkers for PDCI was evaluated by plotting receiver operating characteristic curves and calculating the area under the curve (AUC). Moreover, we assessed the predictive ability of these proteins in combination with neuroimaging. In the discover cohort (n = 100), we identified 25 protein features with best results in the machine learning model, including top-ranked PSAP and H3C15. The two-proteins were used for model construction, achieving an Area under the curve (AUC) of 0.951 in the train set and AUC of 0.981 in the test set. Similarly, the model gives a rank list of endogenous metabolite features, Glycocholic Acid and 6-Methylnicotinamide were two top features. Combining these two markers further got the AUC of 0.969 in train set and 0.867 in the test set. To validate the performance of the protein biomarkers, we performed targeted analysis of selected proteins (H3C15 and PSAP) and proteins likely associated with PDCI (NCAM2 and LAMB2) using parallel reaction monitoring in validation cohort (n = 116). The AUC of the classifier built with H3C15 and PSAP is 0.813. Moreover, when combining H3C15, PSAP, NCAM2, and LAMB2, the model achieved AUC of 0.983 in the train set, AUC of 0.981 in the test set, and AUC of 0.839 in the validation set. Furthermore, we verified that these protein markers we discovered can improve the predictive effect of neuroimaging on PDCI: the classifier built with neuroimaging features had AUC of 0.833, which improved to 0.905 when combined with H3C15. Taken together, our integrated proteomics and metabolomics analysis successfully identified potential biomarkers for PDCI. Additionally, H3C15 showed promise in enhancing the predictive performance of neuroimaging for cognitive impairment.

An evaluation model for an optimal decarbonisation process in the built environment

PurposeThe goal is to deliver a decision-support framework to both public and private entities engaged in energy retrofit investments in the property market.Design/methodology/approachThe evaluation algorithm that is being offered takes an innovative approach to financial and economic analysis. Its foundation is a market-driven/cost-driven method, drawing logic from operational research and goal programming.FindingsThe algorithm is tested to a real estate portfolio yielding an optimal asset retrofitting schedule. The ranking list is determined by taking into consideration a variety of parameters, including investment costs and total CO2 emissions from energy retrofit initiatives. The Carbon Risk Real Estate Monitor’s (CRREM) emission targets for 2030 are employed as a reference point in the process of creating a ranking list of the assets that compose the real estate portfolio under examination.Practical implicationsThe evaluation algorithm will allow to determine, in a real estate portfolio, a priority list of assets to be enhanced. This is accomplished by taking into account the client’s financial resources, the overall cost of the intervention programmes for each asset, and the effects that each asset would have on the environment and the energy once the suggested retrofit programme is put into place.Originality/valueThe study proposes a methodological approach that seeks to balance the optimisation of energy performance, the reduction of environmental effect, the promotion of social well-being and economic sustainability in the context of managing the current property sector.

Determinants of private giving at ranked U.S. universities 2011-2020: A panel data quantile regression analysis

The purpose of this research is to conduct an analysis to identify which variables contribute to student giving to universities in the United States. For this study, data were collected from fifty U.S. universities, most of them in quantile 1 of The Times Higher Education World University Ranking (THE Ranking) list. Also, information from the universities' financial statements and annual reports for the period 2011 to 2020 and information from the National Association of College and University Business Officers (NACUBO). The data were arranged in a panel data structure with a repeated structure for each cross-section. This structure is composed of fifty universities and ten years, as a time period, giving a total of 500 observations. When we perform the corresponding analysis, we can find that the variables that contribute to donations depend on the growth of State GDP, the type of university, university size, state contributions by Full Time Equivalent Students (FTE), the ranking score, tax rebates due to donations. Empirical evidence has also been found in which public universities receive less private giving than private universities. For this research we performed a panel data quantile regression analysis, to more accurately capture the variability of the data because top-ranked universities receive the most donations.

Fast Candidate Screening for Post-diagnosis Refinement

Oftentimes fault candidates produced by logic diagnosis are too many to effectively guide the follow-on failure analysis. In this work, we propose a novel two-stage fast screening method to sift through large amount of candidates in the fault callout outputted by the commercial diagnosis tool. Candidates that are unlikely to be true ones are re-assigned lower ranks or discarded as diagnosis noise. Experimental results based on benchmark designs from various sources show that within the new categorical ranking list, the number of remaining candidates is only 53.78% of the original ones produced by the diagnosis tool, while ensuring an average accuracy of 91.65%, leading to a 1.98 × increase in diagnostic resolution. Our fast screening method only takes seconds of model training and computation, while using no other additional resource besides necessary inputs for generic logic diagnosis.

Internal Medicine Residency Program Applicant Perceptions of the SPLIT Recruitment Model Before and After Universal Virtual Interviews.

Enhancing residency recruitment with modifications to interviews has been an area of national interest, further catalyzed by the transition to universal virtual interviewing (UVI). In 2018, our internal medicine residency program redesigned the recruitment process using virtual interviews. Evaluating this recruitment model allows programs to identify applicant perceptions of each component as they consider enhancements. The new model, termed "SPLIT," included separating optional applicant visit days (AVD) from interviews (S), a pre-interview supplemental form (P), learning program information from a dedicated website (L), virtual interviews (I), and flexible timing (remote interview day and site visit) (T). Applicants for the 2019 to 2023 Match who interviewed at one university-based internal medicine residency program. After rank list certification and before the annual Match, interviewed applicants were surveyed regarding their perceptions of the SPLIT process. Responses before (2019-2020 Matches) and after (2021-2023 Matches) UVIs were compared. A total of 386 (75%) of 515 respondents favored video interviews. This preference was stronger in the post-UVI group (92%) than in the pre-UVI group (57%) (p < 0.001). In total, 76% of respondents attended an AVD (virtual or in-person). Applicants in the post-UVI group also favored having interviews separated from the AVD (p = 0.006) and optional AVDs (p < 0.001), more than those in the pre-UVI group. In the pre-UVI cohort, those who attended an in-person AVD tended to report a higher program understanding (OR 7.8), satisfaction with SPLIT (OR 2.1), and a better recruitment experience (OR 2.0). Virtual interviews were highly rated with increased preference following universal adoption. Optional AVDs separated from virtual interviews enhance applicant understanding of the program and were more effective when offered in-person before the pandemic-related restrictions. As programs begin to reintroduce in-person elements, the SPLIT recruitment model offers an innovative approach that addresses applicant and program needs.

Simulated analogues II: a new methodology for non-parametric matching of models to observations

ABSTRACT Star formation is a multiscale problem, and only global simulations that account for the connection from the molecular cloud-scale gas flow to the accreting protostar can reflect the observed complexity of protostellar systems. Star-forming regions are characterized by supersonic turbulence, and as a result, it is not possible to simultaneously design models that account for the larger environment and in detail reproduce observed stellar systems. Instead, the stellar inventories can be matched statistically, and the best matches found that approximate specific observations. Observationally, a combination of single-dish telescopes and interferometers are now able to resolve the nearest protostellar objects on all scales from the protostellar core to the inner $10\, \mathrm{au}$. We present a new non-parametric methodology which uses high-resolution simulations and post-processing methods to match simulations and observations using deep learning. Our goal is to perform a down-selection from large data sets of synthetic images to a ranked list of best-matching candidates with respect to the observation. This is particularly useful for binary and multiple stellar systems that form in turbulent environments. The objective is to accelerate the rate at which we can do such comparisons, remove biases from hand-picking matches, and contribute to identifying the underlying physical processes that drive the creation and evolution of observed protostellar systems.

Developing a care bundle for children who are eating and drinking with acknowledged risk: A Delphi study with speech and language therapists.

There is minimal research and no formal written guidance for speech and language therapists (SLTs) managing children eating and drinking with acknowledged risks (EDAR). Many SLTs lack confidence due to the complexity of these cases medically, ethically and emotionally. Guidance is recommended to aid paediatric EDAR management. To establish consensus amongst expert paediatric SLTs regarding core components of a care bundle guiding the management of children who are EDAR. A two-round online modified Delphi technique was used. An expert panel of UK SLTs working across paediatric settings (education, hospital, community) were recruited using snowball sampling. Both rounds comprised 36 statements related to potential components of a care bundle, which participants rated on a 7-point scale (strongly agree to strongly disagree), and one open-box question. Consensus was calculated using predetermined criteria (percentage, median and interquartile range) and the final list was ranked by level of importance (mean). A total of 35 participants completed round 1, with 31 completing round 2 (88.6% response rate). Thirty out of 36 statements achieved high to very high consensus. The nine statements reaching very high consensus covered topics such as documentation, capacity, safeguarding and person-centred care. There is a high level of consensus amongst SLTs about core components for a paediatric EDAR care bundle. This study provides a useful starting point for the future development of a care bundle to manage children who are EDAR. What is already known on the subject There is limited research and guidance for SLTs managing children who are EDAR. In adult populations, care bundles have been introduced with positive effects (e.g., better patient outcomes, increased standardisation of care and clinician confidence) and are seen as a promising avenue for paediatrics. What this study adds to the existing knowledge To the research team's knowledge, this is the first published Delphi study on SLT opinion around a care bundle for paediatric EDAR. There were high levels of consensus amongst UK SLTs and the results highlighted documentation, capacity, safeguarding and person-centred care as key areas in practice. Finally, the results provide a ranked list of components UK SLTs felt are important for developing a care bundle. What are the practical and clinical implications of this work? This study could be a useful starting point for creating a care bundle to support the management of children who are EDAR.

Data-driven prioritization of genetic disorders for global genomic newborn screening programs.

Genomic sequencing is poised to expand newborn screening for treatable childhood-onset disorders. Over 30 international research studies and companies are exploring its use, collectively aiming to screen more than 500,000 infants. A key challenge is determining which genes to include in screening. Among 27 newborn sequencing programs, the number of genes analyzed ranged from 134 to 4,299, with only 74 genes included by over 80% of programs. To understand this variability, we assembled a dataset with 25 characteristics of 4,389 genes included in any program and used a multivariate regression analysis to identify characteristics associated with inclusion across programs. These characteristics included presence on the US Recommended Uniform Screening panel, evidence regarding the natural history of disease, and efficacy of treatment. We then used a machine learning model to generate a ranked list of genes, offering a data-driven approach to the future prioritization of disorders for public health newborn screening efforts.

Deciphering digital popularity from question quality: the preeminence of social attributes over textual dimensions in driving CQA platform traffic

ABSTRACT Community Question Answering (CQA) sites, now navigating through millions of daily user visits, hinge significantly on the quality of user-generated questions to bolster user satisfaction and, consequently, website traffic – crucial for platform sustainability. While the quality of questions ostensibly sets the stage for insightful answers, thereby attracting substantial traffic, existing research has predominantly sidelined the significance of questions, favoring a focus on traditional factors, such as answer quality, to elucidate traffic determinants. In this study we quantified the interrelation between question quality and traffic by extracting 15 relevant question-quality features and corresponding traffic (i.e. pageview) data from a leading Chinese CQA platform. Utilizing Grey Relational Analysis, we assessed these attributes’ relative influence on pageviews and created a ranking list of features of questions. Based on the rankings, our findings spotlight the paramount influence of a question's digital popularity and attributes tied to the questioner – central pillars of social features – surpassing the impact of textual elements. This study further delineates the underlying reasons for such patterns based on traffic sourced from diverse channels. Our revelations offer pivotal insights for optimizing CQA platform engagement and furnish avenues for prospective research.

FINEST: Stabilizing Recommendations by Rank-Preserving Fine-Tuning

Modern recommender systems may output considerably different recommendations due to small perturbations in the training data. Changes in the data from a single user will alter the recommendations as well as the recommendations of other users. In applications like healthcare, housing, and finance, this sensitivity can have adverse effects on user experience. We propose a method to stabilize a given recommender system against such perturbations. This is a challenging task due to (1) the lack of a “reference” rank list that can be used to anchor the outputs; and (2) the computational challenges in ensuring the stability of rank lists with respect to all possible perturbations of training data. Our method, FINEST, overcomes these challenges by obtaining reference rank lists from a given recommendation model and then fine-tuning the model under simulated perturbation scenarios with rank-preserving regularization on sampled items. Our experiments on real-world datasets demonstrate that FINEST can ensure that recommender models output stable recommendations under a wide range of different perturbations without compromising next-item prediction accuracy.

Retrieving Continuous Time Event Sequences using Neural Temporal Point Processes with Learnable Hashing

Temporal sequences have become pervasive in various real-world applications such as finance, spatial mobility, health records, etc. Consequently, the volume of data generated in the form of continuous time-event sequence(s) or CTES(s) has increased exponentially in the past few years. Thus, a significant fraction of the ongoing research on CTES datasets involves designing models to address downstream tasks such as next-event prediction, long-term forecasting, sequence classification etc. The recent developments in predictive modeling using marked temporal point processes (MTPP) have enabled an accurate characterization of several real-world applications involving the CTESs. However, due to the complex nature of these CTES datasets, the task of large-scale retrieval of temporal sequences has been overlooked by the past literature. In detail, by CTES retrieval we mean that for an input query sequence, a retrieval system must return a ranked list of relevant sequences from a large corpus. To tackle this, we propose NeuroSeqRet , a first-of-its-kind framework designed specifically for end-to-end CTES retrieval. Specifically, NeuroSeqRet introduces multiple enhancements over standard retrieval frameworks and first applies a trainable unwarping function on the query sequence which makes it comparable with corpus sequences, especially when a relevant query-corpus pair has individually different attributes. Next, it feeds the unwarped query sequence and the corpus sequence into MTPP-guided neural relevance models. We develop four variants of the relevance model for different kinds of applications based on the trade-off between accuracy and efficiency. We also propose an optimization framework to learn binary sequence embeddings from the relevance scores, suitable for the locality-sensitive hashing leading to a significant speedup in returning top-K results for a given query sequence. Our experiments with several datasets show the significant accuracy boost of NeuroSeqRet beyond several baselines, as well as the efficacy of our hashing mechanism.

Program Signaling in Emergency Medicine: The 2022-2023 Program Director Experience.

Program signaling (PS), which enables residency applicants to signal their preference for a specific program, was introduced in emergency medicine (EM) in the 2022-2023 residency application cycle. In this study we evaluated EM program directors' (PD) utilization of PS in application review and ranking. This study also explores the relationship between program characteristics and number of signals received as well as the relative importance and utilization of signals related to the number of signals received. This is an institutional review board-approved, cross-sectional study of PDs at Accreditation Council for Graduate Medical Education-accredited EM residency programs. We used descriptive statistics to describe the characteristics of residency programs and practices around PS. Measures of central tendency and dispersion summarized continuous variables. We used chi-square analysis or the Fisher exact test for comparisons between groups for categorical variables. Comparisons for continuous variables were made using the t-test for independent samples or analysis of variance. The response rate was 41% (n = 113/277 EM programs). Most programs participated in PS (n = 261/277 EM programs, 94.2%). Mean number of signals received was 60 (range 2-203). Signals received varied based on program characteristics including geographic location and program type, duration, environment, and longevity. Most used PS in holistic review (52.2%), but other uses varied by proportion of applications that were signaled. The importance of PS in application review (mean 2.9; 1-5 scale, 1 = not important, 5 = extremely important) and rank list preparation (2.1) was relatively low compared to other application elements such as standardized letters of evaluation (4.97 for review, 4.90 for ranking). The study provides insights into PS utilization in EM's inaugural year. We have identified patterns of signal use based on program characteristics and number of signals received that can inform signal allocation and utilization on an individual applicant and program level. A more nuanced understanding of signal use can provide valuable insight as the specialty of EM grapples with fluctuations in its applicant numbers and shifting demographics of its applicant pool.

GBSR: Graph-based suspiciousness refinement for improving fault localization

Fault Localization (FL) is an important and time-consuming phase of software debugging. The essence of FL lies in the process of calculating the suspiciousness of different program entities (e.g., statements) and generating a ranking list to guide developers in their code inspection. Nonetheless, a prevalent challenge within existing FL methodologies is the propensity for program entities with analogous execution information to receive a similar suspiciousness. This phenomenon can lead to confusion among developers, thereby reducing the effectiveness of debugging significantly. To alleviate this issue, we introduce fine-grained contextual information (such as partial code structural, coverage, and features from mutation analysis) to enrich the characteristics of program entities. Graphical structures are proposed to organize such information, where the passed and failed tests are constructed separately with the consideration of their differential impacts. In order to support the analysis of multidimensional features and the representation of large-scale programs, the PageRank algorithm is adopted to compute each program entity’s weight. Rather than altering the fundamental FL process, we leverage these computed weights to refine the suspiciousness produced by various FL techniques, thereby providing developers with a more precise and actionable ranking of potential fault locations. The proposed strategy Graph-Based Suspiciousness Refinement (GBSR) is evaluated on 243 real-world faulty programs from the Defects4J. The results demonstrate that GBSR can improve the accuracy of various FL techniques. Specifically, for the refinement with traditional SBFL and MBFL techniques, the number of faults localized by the first position of the ranking list (Top-1) is increased by 189% and 68%, respectively. Furthermore, GBSR can also boost the state-of-the-art learning-based FL technique Grace by achieving a 2.8% performance improvement in Top-1.