Analytical Decisions Research Articles

Promises and Pitfalls of Social Media Data Donations

ABSTRACT Studies assessing the effects of social media use are largely based on measures of time spent on social media. In recent years, scholars increasingly ask for more insights in social media activities and content people engage with. Data Download Packages (DDPs), the archives of social media platforms that each European user has the right to download, provide a new and promising method to collect timestamped and content-based information about social media use. In this paper, we first detail the experiences and insights of a data collection of 110 Instagram DDPs gathered from 102 adolescents. We successively discuss the challenges and opportunities of collecting and analyzing DDPs to help future researchers in their consideration of whether and how to use DDPs. DDPs provide tremendous opportunities to get insight in the frequency, range, and content of social media activities, from browsing to searching and posting. Yet, collecting, processing, and analyzing DDPs is also complex and laborious, and demands numerous procedural and analytical choices and decisions.

Evaluating the effects of analytical decisions in large-scale assessments: analyzing PISA mathematics 2003-2012

Research questionThis paper examines the overarching question of to what extent different analytic choices may influence the inference about country-specific cross-sectional and trend estimates in international large-scale assessments. We take data from the assessment of PISA mathematics proficiency from the four rounds from 2003 to 2012 as a case study.MethodsIn particular, four key methodological factors are considered as analytical choices in the rescaling and analysis of the data: (1) The selection of country sub-samples for item calibration differing at three factor levels. (2) The item sample refering to two sets of mathematics items used within PISA. (3) The estimation method used for item calibration: marginal maximum likelihood estimation method as implemented in R package TAM or an pairwise row averaging approach as implemented in the R package pairwise. (4) The type of linking method: concurrent calibration or separate calibration with successive chain linking.FindingsIt turned out that analytical decisions for scaling did affect the PISA outcomes. The factors of choosing different calibration samples, estimation method and linking method tend to show only small effects on the country-specific cross-sectional and trend estimates. However, the selection of different link items seems to have a decisive influence on country ranking and development trends between and within countries.

Exploring the Multiverse of Analytical Decisions in Scaling Educational Large-Scale Assessment Data: A Specification Curve Analysis for PISA 2018 Mathematics Data.

In educational large-scale assessment (LSA) studies such as PISA, item response theory (IRT) scaling models summarize students’ performance on cognitive test items across countries. This article investigates the impact of different factors in model specifications for the PISA 2018 mathematics study. The diverse options of the model specification also firm under the labels multiverse analysis or specification curve analysis in the social sciences. In this article, we investigate the following five factors of model specification in the PISA scaling model for obtaining the two country distribution parameters; country means and country standard deviations: (1) the choice of the functional form of the IRT model, (2) the treatment of differential item functioning at the country level, (3) the treatment of missing item responses, (4) the impact of item selection in the PISA test, and (5) the impact of test position effects. In our multiverse analysis, it turned out that model uncertainty had almost the same impact on variability in the country means as sampling errors due to the sampling of students. Model uncertainty had an even larger impact than standard errors for country standard deviations. Overall, each of the five specification factors in the multiverse analysis had at least a moderate effect on either country means or standard deviations. In the discussion section, we critically evaluate the current practice of model specification decisions in LSA studies. It is argued that we would either prefer reporting the variability in model uncertainty or choosing a particular model specification that might provide the strategy that is most valid. It is emphasized that model fit should not play a role in selecting a scaling strategy for LSA applications.

Assessing research methodologies used to evaluate inequalities in end-of-life cancer care research: a scoping review protocol

IntroductionTo provide equitable cancer care at the end of life, it is essential to first understand the evidence underpinning the existence of unequal cancer outcomes. Study design, measurement and analytical...

Sensitivity Analysis: A Method to Promote Certainty and Transparency in Nursing and Health Research

Nursing and health researchers may be presented with uncertainty regarding the utilization or legitimacy of methodological or analytic decisions. Sensitivity analyses are purposed to gain insight and certainty about the validity of research findings reported. Reporting guidelines and health research methodologists have emphasized the importance of utilizing and reporting sensitivity analyses in clinical research. However, sensitivity analyses are underreported in nursing and health research. The aim of this methodological overview is to provide an introduction to the purpose, conduct, interpretation, and reporting of sensitivity analyses, using a series of simulated and contemporary case examples.

Methodological Considerations in the Application of Multiverse Analyses for More Rigorous Inferences About Nutrition and Health

ObjectivesEvaluating associations between nutritional exposures and disease outcomes requires numerous analytical decisions. Among these decisions are how nutritional exposures are defined and configured, choices and configuration of covariates, participant exclusion criteria, among others. Each decision potentially affects the results and conclusions, yet only a small number of potential models are generally conducted as sensitivity analyses. One approach to evaluate the robustness of conclusions from these myriad decisions is to analyze all reasonable approaches in a so called ‘multiverse analysis’. As part of a project to apply multiverse methodology in nutrition research, we report here some methodological considerations, including the magnitude of covariate selection possibilities.MethodsWe are conducting a multiverse analysis of the association between self-reported beef intake and coronary heart disease (CHD) using the Reasons for Geographic and Racial Differences in Stroke (REGARDS) cohort. We identified common approaches to defining the exposure, outcome, and covariate selection used in the literature to identify reasonable models to incorporate in our multiverse analysis.ResultsCovariates were pooled into 29 distinct concepts (e.g., ‘Physical activity’, ‘Weight’) each which contain a number of possible covariate configurations. For practicality, we limited our analysis to: use of Cox proportional hazards models; one operationalization of CHD; covariate and exposure definitions to a subset of possible configurations for food and nutrients; and excluding linear transformations (e.g., grams of beef versus an 80 gram serving of beef) that do not affect statistical significance. We then varied the number of covariates included and excluded within each model to ensure that covariate concepts were not duplicative, as well as covariate configuration (e.g., continuous, categorical). We estimated 338 quadrillion model specifications with our exposure and covariate lists.ConclusionsResearchers are faced with innumerable choices during model specification in nutrition epidemiology. Multiverse approaches have potential to inform how these choices may influence conclusions from nutrition epidemiological research, but further methodological development is needed.Funding SourcesFunded by the Beef Checkoff

A Substantive Grounded Theory study to explore the basic social process among women with post partum blues in selected tertiary hospitals at Namakkal- A comparative quantitative and qualitative analysis

Context: Grounded theory has been extensively used in research with women, is a means of generating new knowledge that is situated both within the context of the field and existing theory. Methods: The research design used for our study was substantive grounded theory design. The study was conducted in selected tertiary hospitals at Namakkal. The sample size comprises of 5 women with post partum blues. Based on data saturation, the participants were curtailed and extracted with total 5 women with post partum blues. We recruited participants by adopting purposive typical case sampling technique. Typical participants, who were already experiencing and resolving problems related to post partum blues, were enrolled in our study. Demographic variables were collected. The level of post partum blues were measured by Likert Am I blue? Assessment scale. Observations and semi-structured interviews were conducted. All interviews were recorded and transcribed for the women with post partum blues in order to express core variables (basic social process) related to post partum blues with the help of vignettes. Investigator triangulation strategy was used for collecting the data, coding and making analytic decisions. Results: In our study 3 categories were identified viz biological problems, emotional problems and its ramifications and social network support system. The hypothesis generated from our study is the significant correlation between emotional problems and social support among women with post partum blues.

Time to deterioration of symptoms or function using patient-reported outcomes in cancer trials

Time-to-event endpoints for patient-reported outcomes, such as time to deterioration of symptoms or function, are frequently used in cancer clinical trials. Although time-to-deterioration endpoints might seem familiar to cancer researchers for being similar to survival or disease-progression endpoints, there are unique considerations associated with their use. The complexity of time-to-deterioration endpoints should be weighed against the information that they add to the tumour, survival, and safety data used to inform the risks and benefits of an investigational drug. Here we use the estimand framework to show how analytical decisions answer different clinical questions of interest, some of which might be uninformative. Challenges including the consideration of intercurrent events, the difficulty in maintaining adequate completion rates, and considerable patient and trial burden from long-term, serial, patient-reported outcome measurements render time to deterioration a problematic approach for widespread use. For trials in which a comparative benefit in symptoms or function is an objective, an analysis at pre-specified relevant timepoints could be a better approach.

Meta-analysis of single-case experimental designs: How can alternating treatments and changing criterion designs be included?

This text focuses on the quantitative integration of studies using alternating treatments designs (ATDs) and changing criterion designs (CCDs) with other types of single-case experimental designs (SCEDs), such as multiple-baseline and withdrawal, which have received more attention in terms of statistical developments. First, a review of how published meta-analyses have dealt with ATDs and CCDs suggests a variety of analytical strategies and insufficient transparency in reporting the exact comparisons performed. Second, we review data-analytical techniques for ATDs and CCDs, looking for alternatives that can be useful for meta-analysis. Third, without losing sight of the underlying logic of the ATDs and CCDs, we propose using multilevel models, comparing data paths in ATDs and the baseline to the last intervention subphase in CCDs. Furthermore, we suggest additional evaluations of the data pattern, beyond the quantification of the magnitude of effect. We also advocate for transparent reporting of how exactly the conditions are being compared for the different SCEDs. This is possible by specifying the design matrices for the multilevel models used. Fourth, in the context of one of the meta-analyses reviewed, we comment on a series of analytical decisions that need to be made before carrying out a meta-analysis using multilevel models.

How to strengthen the management of acute diverticulitis: the utility of the WSES classification-a prospective single-center observational study.

This study aimed to validate the World Society for Emergent Surgery (WSES) scale for the management of acute left-sided colonic diverticulitis (ALCD). An observational study based on a prospective database of patients with ultrasound (US) and computerized tomography (CT) confirmed ALCD was conducted at our center from April 2018 to May 2019. The primary outcome was the success rate of outpatient management. Secondary outcomes were the association between different WSES stages, clinical and analytical parameters, treatments modalities, and outcomes, and the accuracy of US for management decisions. A total of 230 patients were included. Outpatient management was successful in 51/53 (96.23%) cases with ALCD stage 0 and 62/72 (86.11%) patients with stage 1A. There were no differences in age (p = 0.076) or the presence of pericolic air bubbles (p = 0.06) between patients who underwent admission or outpatient management. Clinical and analytical data, treatment decisions, and outcomes showed statistically significant differences between WSES stages. In 7/12 patients with stage 2A, percutaneous drainage or emergency surgery was required. All cases with stage 2B (distant air) underwent conservative management without the need for emergency or elective surgery. The accuracy of US WSES stages for management decisions, when compared with CT, was 96.96%. The WSES classification for ALCD seemed to be valid helping clinicians in the decision-making process to select between admission or outpatient management. Differences in clinical and analytical data, elected treatments, and outcomes were found between WSES stages. The US WSES stages showed high accuracy for management decisions.

Trends in the Annual Consultation Incidence and Prevalence of Low Back Pain and Osteoarthritis in England from 2000 to 2019: Comparative Estimates from Two Clinical Practice Databases.

PurposeTo compare estimates of annual person-consulting incidence and prevalence of low back pain (LBP) and osteoarthritis for two national English electronic health record databases (Clinical Practice Research Datalink (CPRD) Aurum and CPRD GOLD).Patients and MethodsRetrospective, population-based, longitudinal cohort study. LBP and osteoarthritis cases were defined using established codelists in people aged ≥15 and ≥45 years, respectively. Incident cases were new recorded cases in a given calendar year with no relevant consultation in the previous 3 years (denominator = exact person-time in the same calendar year for the at-risk population). Prevalent cases were individuals with ≥1 consultation for the condition of interest recorded in a given calendar year, irrespective of prior consultations for the same condition (denominator = all patients with complete registration history in the previous 3 years). We estimated age-sex standardised incidence and annual (12-month period) prevalence for both conditions in 2000–2019, overall, and by sex, age group, and region.ResultsStandardised incidence and prevalence of LBP from Aurum were lower than those from GOLD until 2014, after which estimates were similar. Both databases showed recent declines in incidence and prevalence of LBP: declines began earlier in GOLD (after 2012–2014) than Aurum (after 2014–2015). Standardised incidence (after 2011) and prevalence of osteoarthritis (after 2003) were higher in Aurum than GOLD and showed different trends: incidence and prevalence were stable or increasing in Aurum, decreasing in GOLD. Stratified estimates in CPRD Aurum suggested consistently higher occurrence among women, older age groups, and those living in the north of England.ConclusionComparative analyses of two English databases produced conflicting estimates and trends for two common musculoskeletal conditions. Aurum estimates appeared more consistent with external sources and may be useful for monitoring population musculoskeletal health and healthcare demand, but they remain sensitive to analytic decisions and data quality.

"You tell him that 'baby, I am protecting myself'": Women's agency and constraint around willingness to use pre-exposure prophylaxis in the Masibambane Study.

Objectives:To explore women’s willingness to consider using pre-exposure prophylaxis for HIV prevention in the context of gendered relationship dynamics, in Durban, South Africa.Methods:As formative research prior to development of a gender-informed intervention to introduce pre-exposure prophylaxis to young, urban, educated women, we conducted six focus-group discussions and eight in-depth interviews with 46 women ages 18–25 years, who were not current pre-exposure prophylaxis users. Women were recruited from clinic and community settings using a criterion-based snowball sampling technique. Qualitative data were coded and analyzed thematically, with a team-based consensus approach for final coding, analytical decisions, and data interpretation.Results:Women clearly understood the benefits of pre-exposure prophylaxis for themselves and their partners, focusing on promoting health and their right to protect themselves from HIV infection. At the same time, and in accordance with findings from other studies, women were realistic about the concerns that would arise among male partners, including disapproval, loss of trust, possible loss of the relationship, and in some instances, the potential for violence, if they were to propose pre-exposure prophylaxis use. To resolve this tension, some women advocated for covert use as the best option for themselves and others argued for disclosure, proposing various approaches to working with partners to adopt pre-exposure prophylaxis. The suggestion that both partners use pre-exposure prophylaxis was made repeatedly. Thus, women sought to avoid discussions of trust or lack of trust and a partner’s possible infidelities, choosing instead to focus on preserving or even building a relationship through suggesting pre-exposure prophylaxis use.Conclusion:Women offered diverse narratives on agency and constraint in relation to choosing pre-exposure prophylaxis as a future prevention strategy, as well as ways to engage with their male partners about pre-exposure prophylaxis. These findings speak to the need for interventions to bolster women’s confidence, sense of empowerment, and their communication and decision-making skills for successful HIV prevention.

Hormones, ovulatory cycle phase and pathogen disgust: A longitudinal investigation of the Compensatory Prophylaxis Hypothesis

Multiple studies have argued that disgust, especially pathogen disgust and contamination sensitivity, change across women's ovulatory cycle, with higher levels in the luteal phase due to an increase in progesterone levels. According to the Compensatory Prophylaxis Hypothesis (CPH), women have a higher disgust sensitivity to pathogen cues when in the luteal phase (or when progesterone levels are higher), because progesterone is associated with suppressed immune responses. Evidence for this hypothesis is rather mixed and uncertain, as the largest study conducted so far reported no compelling evidence for an association between progesterone levels and pathogen disgust. Further, ovulatory cycle research has been criticized for methodological shortcomings, such as invalid cycle phase estimates, no direct hormone assessments, small sample sizes or between-subjects studies. To address these issues and to contribute to the literature, we employed a large, within-subjects design (N = 257 with four sessions each), assessments of salivary hormone levels and cycle phase estimates based on luteinizing hormone tests. A variety of multilevel models suggest no compelling evidence that self-reported pathogen disgust or contamination sensitivity is upregulated in the luteal phase or tracks changes in women's hormone levels. We further found no compelling evidence for between-subjects associations of pathogen disgust or contamination sensitivity and hormone levels. Results remain robust across different analytical decisions (e.g. in a subsample of women reporting feeling sick). We discuss explanations for our results, limitations of the current study and provide directions for future research.

Pandemic visitor policies: Parent reactions and policy implications.

Pandemic visitor policies: Parent reactions and policy implications.

Numerical uncertainty in analytical pipelines lead to impactful variability in brain networks

The analysis of brain-imaging data requires complex processing pipelines to support findings on brain function or pathologies. Recent work has shown that variability in analytical decisions, small amounts of noise, or computational environments can lead to substantial differences in the results, endangering the trust in conclusions. We explored the instability of results by instrumenting a structural connectome estimation pipeline with Monte Carlo Arithmetic to introduce random noise throughout. We evaluated the reliability of the connectomes, the robustness of their features, and the eventual impact on analysis. The stability of results was found to range from perfectly stable (i.e. all digits of data significant) to highly unstable (i.e. 0 - 1 significant digits). This paper highlights the potential of leveraging induced variance in estimates of brain connectivity to reduce the bias in networks without compromising reliability, alongside increasing the robustness and potential upper-bound of their applications in the classification of individual differences. We demonstrate that stability evaluations are necessary for understanding error inherent to brain imaging experiments, and how numerical analysis can be applied to typical analytical workflows both in brain imaging and other domains of computational sciences, as the techniques used were data and context agnostic and globally relevant. Overall, while the extreme variability in results due to analytical instabilities could severely hamper our understanding of brain organization, it also affords us the opportunity to increase the robustness of findings.

The Effect of Individual Analytical Orientation and Capabilities on Decision Quality and Regret

Decision makers are exposed to an increasing amount of information. Algorithms can help people make better data-driven decisions. Previous research has focused on both companies’ orientation towards analytics use and the required skills of individual decision makers. However, each individual can make either analytically based or intuitive decisions. We investigated the characteristics that influence the likelihood of making analytical decisions, focusing on both analytical orientation and capabilities of individuals. We conducted a survey using 462 business students as proxies for decision makers and used partial least squares path modeling to show that analytical capabilities and analytical orientation influence each other and affect analytical decision-making, thereby impacting decision quality and decision regret. Our findings suggest that when implementing business analytics solutions, companies should focus on the development not only of technological capabilities and individuals’ skills but also of individuals’ analytical orientation.

A comorbid mental disorder paradox: Using causal diagrams to understand associations between posttraumatic stress disorder and suicide.

Although some studies document that posttraumatic stress disorder (PTSD) increases suicide risk, other studies have produced the paradoxical finding that PTSD decreases suicide risk. We sought to understand methodologic biases that may explain these paradoxical findings through the use of directed acyclic graphs (DAGs). DAGs are causal diagrams that visually encode a researcher's assumptions about data generating mechanisms and assumed causal relations among variables. DAGs can connect theories to data and guide statistical choices made in study design and analysis. In this article, we describe DAGs and explain how they can be used to identify biases that may arise from inappropriate analytic decisions and data limitations. We define a particular form of bias, collider bias, that is a likely explanation for why studies have found a supposedly protective association of PTSD with suicide. This protective association is interpreted by some researchers as evidence that PTSD reduces the risk of suicide. Collider bias may occur through inappropriate adjustment for a psychiatric comorbidity, such as adjustment for variables that are affected by PTSD and share common causes with suicide. We recommend that researchers collect longitudinal measurements of psychiatric comorbidities, which would help establish the temporal ordering of variables and avoid the biases discussed in this article. Furthermore, researchers could use DAGs to explore how results may be impacted by design and analytic decisions prior to execution. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

165 'What Matters' in the Emergency Department: Older Adults' Desires and Clinicians' Beliefs

One model germane to ED geriatric care is the 4Ms model – What Matters, Medications, Mentation, and Mobility – developed via the Age-Friendly Health Systems initiative. A feasible approach to identifying ‘What Matters’ in the ED is lacking. Therefore, we sought to describe older adult patient priorities when seeking emergency care and to identify the anticipated disposition concordance with their treating clinicians. We conducted a qualitative and quantitative analysis of cognitively intact patients aged 65 years and older receiving care in two EDs and their treating clinicians. Stakeholders previously developed the ‘What Matters in the ED’ structured conversation guide to explore patients’ concerns and expectations; a question about anticipated disposition was added. Participants were interviewed during the ED encounter when disposition uncertainty still existed. The first question ascertained patients’ concerns and fears about their health and healthcare; the second question identified the outcome patients most wanted from their ED visit. The final question asked, “Do you believe that hospitalization will help you today? If so, why?” We asked analogous structured questions to the patient’s treating clinician regarding the patients’ priorities. Interviews were professionally transcribed and an inductive approach of thematic analysis was used by two authors (CJG, HD) to line-by-line review and analyze interview data. We created a code key, grouped codes into themes, and maintained an audit trail of all analytic decisions. Of the 51 patients approached, 40 persons aged 65+ and their emergency clinicians completed interviews. Patient interviews ranged in time from 1 to 8 minutes (mean - 2 minutes, 50 seconds). Three major themes among participants included: 1) Older adults seeking emergency care were concerned regarding preservation of current status, contributing factors related to chronic conditions, and symptom resolution, whereas clinicians almost exclusively referenced the chief complaint as the patient’s greatest concern, 2) Older adults valued diagnosis and symptom resolution, followed closely by their ability to ‘go home’ as outcomes sought, with clinicians also commonly identifying disposition as a primary concern to the older adult, and 3) Older adults often believed that hospital admission was warranted, whereas clinicians often felt that discharge was the appropriate disposition. Quantitatively, patient-clinician anticipated disposition concordance was present in 60% of encounters (Cohen’s kappa coefficient 0.217), suggesting fair agreement. Older adults identified several concerns regarding their health in comparison to their treating clinicians, suggesting emergency clinicians may be unaware of certain patient priorities during the encounter. Despite often being in alignment with desired patient outcomes, older adults and clinicians exhibited only fair agreement in anticipated disposition. This work identifies the feasibility of incorporating the ‘What Matters’ questions in the ED, and the importance of aligning emergency care with the value-based priorities of older adults.

Best practices in statistical computing.

The world is becoming increasingly complex, both in terms of the rich sources of data we have access to and the statistical and computational methods we can use on data. These factors create an ever-increasing risk for errors in code and the sensitivity of findings to data preparation and the execution of complex statistical and computing methods. The consequences of coding and data mistakes can be substantial. In this paper, we describe the key steps for implementing a code quality assurance (QA) process that researchers can follow to improve their coding practices throughout a project to assure the quality of the final data, code, analyses, and results. These steps include: (i) adherence to principles for code writing and style that follow best practices; (ii) clear written documentation that describes code, workflow, and key analytic decisions; (iii) careful version control; (iv) good data management; and (v) regular testing and review. Following these steps will greatly improve the ability of a study to assure results are accurate and reproducible. The responsibility for code QA falls not only on individual researchers but institutions, journals, and funding agencies as well.

Understanding the effects of conceptual and analytical choices on ‘finding’ the privacy paradox: A specification curve analysis of large-scale survey data

ABSTRACT The privacy paradox suggests that privacy concerns do not relate to privacy-related behavior. Although it has inspired numerous studies, findings remain inconclusive. Some of the inconsistencies in published findings may be explained by a strong heterogeneity in the conceptual and analytical choices that researchers implement when investigating the privacy paradox. Based on representative survey data of the 27 EU member states (2011: n = 8,962; 2015: n = 10,526; 2019: n = 11,428), I investigated the effect of conceptual and analytical decisions on ‘finding’ the privacy paradox. Specification curve analyses revealed that the magnitude and statistical significance of the relationship between privacy concerns and information disclosure is contingent on the operationalization of the independent variable, the inclusion of covariates, and the age of the studied population. The relationship between online privacy concerns and using social media privacy settings, in contrast, was less influenced by analytical decisions. Yet, the relationship was stronger in younger people and increased over time. The findings call for more transparency in analyzing research data. Evaluating the implications of analytical choices will help to establish best practices and advance cumulative knowledge creation in privacy research.