Missing Data Research Articles

The Effect of the Apollo Neuro Device on Anxiety Among Participants who Underwent Ketamine Assisted Therapy

IntroductionThe study aimed to evaluate the effectiveness of a device called “Apollo” in reducing anxiety, as compared to a control group. Participants were divided into two groups: the intervention group (receiving the “Apollo” device) and the control group (receiving no intervention).ObjectivesThe primary outcome measure was the change in Generalized Anxiety Disorder 7-item (GAD-7) scores, calculated as the difference between post-GAD-7 and pre-GAD-7 scores.MethodsParticipants were recruited from two different cohorts, with the intervention group derived from the “Apollo” dataset and the control group derived from the “KaT Cohort 9” dataset. Matching was performed based on Age, Sex, and pre-GAD-7 scores to create comparable groups.For those with full datasets, a total of 4 (out of 5) participants from the “Apollo” group were matched with 15 (out of 45) participants from the “Control” group, based on the selected criteria. Data cleaning was performed to handle missing values and non-numeric entries. Propensity score matching was used to match participants from the “Apollo” and “Control” groups based on Age, Sex, and pre-GAD-7 scores. An independent samples t-test was conducted to compare the mean change in GAD-7 scores between the two groups. Since propensity score matching requires complete data on matching factors (age, sex, pre-GAD-7), those without full datasets were excluded.ResultsThe median change in GAD-7 scores in the “Apollo” group was −8.5, indicating a median reduction in anxiety symptoms.The independent samples t-test revealed no statistically significant difference in the change in GAD-7 scores between the “Apollo” and “Control” groups (t = -0.889, p = 0.387). Therefore, the study so far did cannot conclude a significant difference.ConclusionsKetamine assisted therapy remains a promising way to decrease anxiety among patients with generalized anxiety disorder and elevated GAD-7 scores. Ways to potentially improve these results are increasing the number of Apollo patients and having more balanced numbers between groups.Disclosure of InterestNone Declared

Symptom-Based Disease Prediction: A Machine Learning Approach

The advent of machine learning techniques has revolutionized various sectors, including healthcare. This project concentrates on leveraging machine learning algorithms for disease prediction based on symptoms. With a dataset comprising 132 symptoms and 41 diseases, the aim is to develop a robust predictive model capable of accurately diagnosing diseases given a set of symptoms. The process involves several key steps. Initially, the dataset is preprocessed to handle missing values, encode categorical variables, and normalize the data. To determine which symptoms are most pertinent to the prognosis of a disease, feature selection techniques are utilized. Various machine learning algorithms, including decision trees, support vector machines, random forests, and XGBoost, were explored to determine the most effective prediction model. XGBoost, in particular, emerges as one of the topperforming models because of its capacity to manage complicated relationships within the data and its effectiveness in handling imbalanced datasets. To evaluate the models' performance, evaluation criteria like accuracy, precision, recall, and F1-score are used. Moreover, to enhance model performance and avoid overfitting, techniques like crossvalidation and hyperparameter tuning are utilized. The proposed system holds significant potential in aiding healthcare professionals in diagnosing diseases promptly and accurately, thereby improving patient outcomes and reducing healthcare costs. It is important to note that the model needs further validation on diverse datasets and regular updates to remain relevant in clinical settings.

Identification Method of High Incidence Areas of Bad Driving Behaviors Based on DBSCAN-Graham Combination

<p>This study utilizes data obtained from vehicle positioning, speed, and engine status through satellite positioning system monitoring terminals. The data is processed using spherical interpolation of latitude and longitude to handle missing values. The study establishes indicators for identifying bad driving behaviors, including speeding, rapid acceleration and deceleration, prolonged vehicle idling, and fatigue driving. The traditional DBSCAN algorithm is improved by incorporating both temporal and spatial search ranges into the algorithm, enabling cluster analysis of bad driving behaviors from both temporal and geographical perspectives. Finally, the Graham algorithm is utilized to calculate cluster boundaries, determining the specific locations and boundaries of high incidence areas of bad driving behaviors. This research aids regulatory authorities in more precisely supervising risk areas during transportation processes in terms of time and location.</p> <p> </p>

A Randomized Clinical Trial Comparing the Effects of Mindfulness-Based and Cognitive Behavioral Therapy-Based Stress Reduction in Medical Students

IntroductionMedical students face an enormous amount of stress (Dyrbye LN et al. Ann Intern Med 2008; 149: 334-41). They suffer from higher rates of depression, anxiety, and suicide compared to the general population. Despite experiencing more mental health problems, there is a lack of research exploring ways to improve their mental health. Although there are a few small sample studies investigating the effectiveness of Mindfulness-Based Stress Reduction (MBSR) on medical students, there is no study comparing its effectiveness against an active intervention group in the literature (van Dijk I et al. Acad Med 2017; 92: 1012-1021)ObjectivesWe aimed to compare the effects of the Mindfulness-Based Stress Reduction (MBSR) and the Cognitive Behavioural Based Stress Reduction (CBSR) group interventions on depressive and anxious symptoms and perceived stress of medical students.Methods323 medical students applied to participate in one of the group interventions and were assessed with the Mini International Neuropsychiatric Interview. Of these, 253 (77% female, mean age=21.9 ± 2.9 years) were allocated into online MBSR (n=127) and online CBSR (n=126) groups after randomization. Their anxiety and depressive symptoms and perceived stress levels were assessed at baseline and after 8 weeks of interventions. 33,2% of participants (MBSR: n=39; CBSR: n=45) completed the protocol by attending five or more sessions. Both intention-to-treat (ITT) analysis and per-protocol (PP) analysis were used to assess outcomes. In the ITT analysis, we used multiple imputation to address missing values. All assessments and group interventions were done online.ResultsIn the ITT analysis, both MBSR and CBSR were found to be slight to moderately effective in reducing symptoms of depression (MBSR: d=.50; CBSR: d=.40), anxiety (MBSR:d=.73; CBSR: d=.52), and perceived stress (MBSR: d=.48; CBSR: d=.42), but they were no superior to each other. In the PP analysis, both interventions moderately to strongly improved the symptoms of depression (MBSR: d=1.03; CBSR: d=.74), anxiety (MBSR: r=-.74; CBSR: r=-.72), and perceived stress (MBSR: r=-.80; CBSR: r=-.68). While there was no statistically significant difference between them in reducing depressive symptoms and perceived stress, MBSR was found to be significantly more effective than CBSR in reducing anxiety symptoms (u=469, z=-2.756, p=0.006).ConclusionsBoth MBSR and CBSR improve symptoms of depression and anxiety in medical students after 8 weeks of interventions. Completing the protocol or attending more sessions may increase the effectiveness of the interventions. While the interventions did not show superiority to each other in terms of effectiveness in reducing depressive symptoms and perceived stress, MBSR appears to be more effective in reducing anxiety symptoms compared to CBSR in the group that completed the protocol.Disclosure of InterestNone Declared

Beyond Collaborative Filtering: A Relook at Task Formulation in Recommender Systems

Recommender Systems (RecSys) have become indispensable in numerous applications, profoundly influencing our everyday experiences. Despite their practical significance, academic research in RecSys often abstracts the formulation of research tasks from real-world contexts, aiming for a clean problem formulation and more generalizable findings. However, it is observed that there is a lack of collective understanding in RecSys academic research. The root of this issue may lie in the simplification of research task definitions, and an overemphasis on modeling the decision outcomes rather than the decision-making process. That is, we often conceptualize RecSys as the task of predicting missing values in a static user-item interaction matrix, rather than predicting a user's decision on the next interaction within a dynamic, changing , and application-specific context. There exists a mismatch between the inputs accessible to a model and the information available to users during their decision-making process, yet the model is tasked to predict users' decisions. While collaborative filtering is effective in learning general preferences from historical records, it is crucial to also consider the dynamic contextual factors in practical settings. Defining research tasks based on application scenarios using domain-specific datasets may lead to more insightful findings. Accordingly, viable solutions and effective evaluations can emerge for different application scenarios.

Визуализация в задачах реконструкции данных

Many application tasks of multidimensional data analysis which describe the state of real physical or other systems face with difficulties. This is a consequence of the low-quality source data, including missing values, the probability of errors or unreliability of measurements. Incomplete data can become an obstacle for research using many modern informational methods. The current work examines the potential and capabilities of visual analytics tools for preliminary preparation, correction or complete analysis of primary data volumes. A promising area of application of the approach discussed in the study is the targeted use of visualization capabilities as a data analysis tool. The implementation of specialized visual metaphors is used to solve problems of processing and interpreting data, the sources of which are cyberphysical systems of different complexity levels. Such systems operate in an autonomous or partially controlled mode. A characteristic feature of these systems is the presence of a large number of sensors that collect various types of data. Such data differ in the capacity of the corresponding information channels, their speed and reliability. Examples of such cyberphysical systems are unmanned aerial vehicles (UAVs), robotic stations, and multimodal monitoring systems. These systems can function in conditions where it is difficult to obtain objective observation experience (deep-sea robots). The effective use of data collected by cyberphysical monitoring systems is a condition for solving a large number of application and research tasks.

Effect of Brand Elements on Organizational Success: A Moderating Approach to the Brand Experience of Muslim and Non-Muslim Customers of Islamic financial Products in Pakistan

The primary purpose of this research is to analyze the impact of brand elements on the achievement of an organization. In addition, the study also examined the moderating effect of brand experience on Muslim and non-Muslim customers' perception of Islamic products in Pakistan. This research examines the significance of brand elements in the success of organizations and compares the reactions of Muslim and Non-Muslim customers of Islamic Products in Pakistan. The researchers used Keller's renowned CBBE model to examine the impact of brand elements on organizational success. This research used the convenience sampling technique to gather data from customers of Islamic products. Following the completion of the missing value analysis, only 124 out of 150 respondents provided their replies. The study analysis indicates that the brand resonance of both Muslim and non-Muslim customers of Islamic products has a positive effect on the success of the organization. Additionally, the relationship between these consumers and the organization is also crucial for achieving success or improved performance. The study shows that both Muslim and non-Muslim consumers in Pakistan have positive responses towards Islamic products offered by Islamic banks. This information can be used by managers to implement this strategy for the success of their organizations.

Exploring the Role of Deep Learning in Forecasting for Sustainable Development Goals: A Systematic Literature Review

This paper aims to explore the relationship between deep learning and forecasting within the context of the Sustainable Development Goals (SDGs). The primary objective is to systematically review 38 articles published between 2019 and 2023, following PRISMA guidelines, to understand the current landscape of deep learning forecasting for SDGs. Using data from 2019-2023 allows capturing the latest developments in deep learning forecasting for Sustainable Development Goals (SDGs), while excluding data before 2019 and after 2023 is based on the desire to avoid including potentially less relevant or unpublished research and to maintain focus on the most current and contextually relevant literature. The methodological approach involves analyzing the application of deep learning methods for forecasting within various SDG fields and identifying trends, challenges, and opportunities. The literature review results reveal the popularity of LSTM models, challenges related to data availability, and the interconnected nature of SDGs. Additionally, the study demonstrates that deep learning models enhance forecast accuracy and computational performance, as measured by Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE), and R-squared (R2). The findings underscore the importance of advanced data preparation techniques and the integration of deep learning with SDGs to improve forecasting outcomes. The novelty of this research lies in its comprehensive overview of the current landscape and its valuable insights for researchers, policymakers, and stakeholders interested in advancing sustainable development goals through deep learning forecasting. Finally, the paper suggests future research directions, including exploring the potential of hybrid forecasting models and investigating the impact of emerging technologies on SDG forecasting methodologies. Innovative methods for imputing missing values in deep learning forecasting models could be further explored to enhance predictive accuracy and robustness.

RELTRAD2: Refining the State of the Art of Religious Classification by Reconsidering the Categorization of Nondenominational Respondents

AbstractRELTRAD is a major religious taxonomy used by a large number of researchers. Although criticisms have been raised about its utility, improving the algorithm to capture contemporary religious dynamics is important given its widespread use. The present RELTRAD taxonomy classifies more religiously active nondenominational respondents as Conservative Protestants and codes the remainder as missing data. A growing number of Americans indicate they are either nondenominational or only Christian or Protestant, which means using RELTRAD in its existing form codes a nonrandom and increasingly large number of respondents with a missing value for religious affiliation (growing from 2 percent to 5 percent of the US General Social Survey (GSS) sample between 2000 and 2018). Using a machine learning algorithm to predict the likely religious tradition of nondenominational respondents, we demonstrate the shortcomings of this approach and introduce a new coding scheme, RELTRAD2, which classifies nondenominational respondents who report a Black racial identity as Black Protestant, non‐Black respondents who never attend religious services as Mainline Protestant, and the remainder as Conservative Protestant. Code to derive RELTRAD2 from the GSS is provided.

Strategies for Imputing Missing Values and Removing Outliers in the Dataset for Machine Learning-Based Construction Cost Prediction

Accurately predicting construction costs during the initial planning stages is crucial for the successful completion of construction projects. Recent advancements have introduced various machine learning-based methods to enhance cost estimation precision. However, the accumulation of authentic construction cost data is not straightforward, and existing datasets frequently exhibit a notable presence of missing values, posing challenges to precise cost predictions. This study aims to analyze diverse substitution methods for addressing missing values in construction cost data. Additionally, it seeks to evaluate the performance of machine learning models in cost prediction through the removal of conditional outliers. The primary goal is to identify and propose optimal strategies for handling missing value in construction cost records, ultimately improving the reliability of cost predictions. According to the analysis results, among single imputation methods, median imputation emerges as the most suitable, while among multiple imputation methods, lasso regression imputation produces the most superior outcomes. This research contributes to enhancing the trustworthiness of construction cost predictions by presenting a pragmatic approach to managing missing data in construction cost performance records, thereby facilitating more precise project planning and execution.

Exploiting nearest neighbor data and fuzzy membership function to address missing values in classification.

The accuracy of most classification methods is significantly affected by missing values. Therefore, this study aimed to propose a data imputation method to handle missing values through the application of nearest neighbor data and fuzzy membership function as well as to compare the results with standard methods. A total of five datasets related to classification problems obtained from the UCI Machine Learning Repository were used. The results showed that the proposed method had higher accuracy than standard imputation methods. Moreover, triangular method performed better than Gaussian fuzzy membership function. This showed that the combination of nearest neighbor data and fuzzy membership function was more effective in handling missing values and improving classification accuracy.

Use of Sequential Hot-Deck Imputation for Missing Health Care Systems Data for Population Health Research.

Electronic medical record (EMR) data present many opportunities for population health research. The use of EMR data for population risk models can be impeded by the high proportion of missingness in key patient variables. Common approaches like complete case analysis and multiple imputation may not be appropriate for some population health initiatives that require a single, complete analytic data set. In this study, we demonstrate a sequential hot-deck imputation (HDI) procedure to address missingness in a set of cardiometabolic measures in an EMR data set. We assessed the performance of sequential HDI within the individual variables and a commonly used composite risk score. A data set of cardiometabolic measures based on EMR data from 2 large urban hospitals was used to create a benchmark data set with simulated missingness. Sequential HDI was applied, and the resulting data were used to calculate atherosclerotic cardiovascular disease risk scores. The performance of the imputation approach was assessed using a set of metrics to evaluate the distribution and validity of the imputed data. Of the 567,841 patients, 65% had at least 1 missing cardiometabolic measure. Sequential HDI resulted in the distribution of variables and risk scores that reflected those in the simulated data while retaining correlation. When stratified by age and sex, risk scores were plausible and captured patterns expected in the general population. The use of sequential HDI was shown to be a suitable approach to multivariate missingness in EMR data. Sequential HDI could benefit population health research by providing a straightforward, computationally nonintensive approach to missing EMR data that results in a single analytic data set.

Hybrid Data Augmentation Based Machine Learning Approach for Botnet Attack Detection in IOT Networks

This paper presents a comprehensive approach to botnet detection in Internet of Things (IoT) networks through the development and evaluation of a Generative Adversarial Network (GAN) augmented machine learning model. The methodology encompasses a multi-step process, starting with data collection and pre-processing, including feature extraction, normalization, and handling missing values. To address the challenge of data imbalance, a novel application of GANs is proposed. For classification of network traffic into botnet and legitimate traffic is performed using xgboost. The performance of the proposed model is rigorously evaluated using the N-BaIoT dataset, demonstrating its effectiveness through high accuracy, precision, recall, and F1-score metrics. The results indicate significant improvements over existing models, showcasing the potential of the proposed methodology in enhancing IoT network security against botnet threats.

Masked Self-Attention Dynamic Imputation Model for Physiological Indicators

Accurate imputation of missing and outlier values in intraoperative physiological indicators can assist doctors in swiftly taking measures to prevent risks during surgery, which in turn helps improve postoperative prognosis for patients. Traditional data imputation methods have focused too much on the current data itself, neglecting the contextual information provided by surrounding data points. Addressing this issue, the imputation process for intraoperative physiological indicator data obtained through monitoring has been enhanced with a model that employs a self-attention mechanism focusing on the data context for dynamic weighting during imputation. Experimental results indicate that the proposed model achieves a MAE of 2.06%, a RMSE of 7.08%, and a MRE of 2.8%, outperforming other comparative models.

An Improved Data Interpolating Empirical Orthogonal Function Method for Data Reconstruction: A Case Study of the Chlorophyll-a Concentration in the Bohai Sea, China

Chlorophyll-a (chl-a) serves as a key indicator in water quality and harmful algal blooms (HABs) research. While satellite ocean color data have greatly advanced chl-a research and HABs monitoring, missing data caused by cloud cover and other factors limit the spatiotemporal continuity and the utility of remote sensing data products. The Data Interpolating Empirical Orthogonal Function (DINEOF) method, widely used to reconstruct missing values in remote sensing datasets, is open to improvement in terms of computational accuracy and efficiency. We propose an improved method called Concentration-Stratified DINEOF (CS-DINEOF), which uses a coordinate–value correlative data division strategy to stratify the study area into several subregions based on annual average chl-a concentration. The proposed method clusters data points with similar spatiotemporal patterns, allowing for more targeted and effective reconstruction in each sub-dataset. The feasibility and advantage of the proposed method are tested and evaluated in the experiments of chl-a data reconstruction in the water of the Bohai Sea. Compared with the ordinary DINEOF method, the CS-DINEOF method improves the reconstruction accuracy, with an average Root Mean Square Error (RMSE) reduction of 0.0281 mg/m3, and saves computational time by 228.9%. Furthermore, the gap-free images generated from CS-DINEOF are able to illustrate small variations and details of the chl-a distribution in local areas. We can conclude that the proposed CS-DINEOF method is superior in providing significant insights for water quality and HABs studies in the Bohai Sea region.

Assessment of fetal corpus callosum biometry by 3D super-resolution reconstructed T2-weighted magnetic resonance imaging.

To assess the accuracy of corpus callosum (CC) biometry, including sub-segments, using 3D super-resolution fetal brain MRI (SR) compared to 2D or 3D ultrasound (US) and clinical low-resolution T2-weighted MRI (T2WS). Fetal brain biometry was conducted by two observers on 57 subjects [21-35 weeks of gestational age (GA)], including 11 cases of partial CC agenesis. Measures were performed by a junior observer (obs1) on US, T2WS and SR and by a senior neuroradiologist (obs2) on T2WS and SR. CC biometric regression with GA was established. Statistical analysis assessed agreement within and between modalities and observers. This study shows robust SR to US concordance across gestation, surpassing T2WS. In obs1, SR aligns with US, except for genu and CC length (CCL), enhancing splenium visibility. In obs2, SR closely corresponds to US, differing in rostrum and CCL. The anterior CC (rostrum and genu) exhibits higher variability. SR's regression aligns better with literature (US) for CCL, splenium and body than T2WS. SR is the method with the least missing values. SR yields CC biometry akin to US (excluding anterior CC). Thanks to superior 3D visualization and better through plane spatial resolution, SR allows to perform CC biometry more frequently than T2WS.

A Matrix Completion Method for Imputing Missing Values of Process Data

Real-time process data are the foundation for the successful implementation of intelligent manufacturing in the chemical industry. However, in the actual production process, process data may randomly be missing due to various reasons, thus affecting the practical application of intelligent manufacturing technology. Therefore, this paper proposes the application of appropriate matrix completion algorithms to impute the missing values of real-time process data. Considering the characteristics of online missing value imputation problems, this paper proposes an improved method for a matrix completion algorithm that is suitable for real-time missing data imputation. By utilizing real device data, this paper studies the impact of algorithm parameters on the effect of missing value imputing and compares it with several classical missing value imputing methods. The results show that the introduced method achieves higher accuracy in data imputation compared to the baseline method. Furthermore, the proposed enhancement significantly improves the speed performance of algorithms.

Provision of Medications for Self-Managed Abortion Before and After the Dobbs v Jackson Women’s Health Organization Decision

The Supreme Court decision in Dobbs v Jackson Women's Health Organization overturned the right to choose abortion in the US, with at least 16 states subsequently implementing abortion bans or 6-week gestational limits. Prior research indicates that in the 6 months following Dobbs, approximately 32 360 fewer abortions were provided within the US formal health care setting. However, trends in the provision of medications for self-managed abortion outside the formal health care setting have not been studied. To determine whether the provision of medications for self-managed abortion outside the formal health care setting increased in the 6 months after Dobbs. Cross-sectional study using data from sources that provided abortion medications outside the formal health care setting to people in the US between March 1 and December 31, 2022, including online telemedicine organizations, community networks, and online vendors. Using a hierarchical bayesian model, we imputed missing values from sources not providing data. We estimated the change in provision of medications for self-managed abortion after the Dobbs decision. We then estimated actual use of these medications by accounting for the possibility that not all provided medications are used by recipients. Abortion restrictions following the Dobbs decision. Provision and use of medications for a self-managed abortion. In the 6-month post-Dobbs period (July 1 to December 31, 2022), the total number of provisions of medications for self-managed abortion increased by 27 838 (95% credible interval [CrI], 26 374-29 175) vs what would have been expected based on pre-Dobbs levels. Excluding imputed data changes the results only slightly (27 145; 95% CrI, 25 747-28 246). Accounting for nonuse of medications, actual self-managed medication abortions increased by an estimated 26 055 (95% CrI, 24 739-27 245) vs what would have been expected had the Dobbs decision not occurred. Provision of medications for self-managed abortions increased in the 6 months following the Dobbs decision. Results suggest that a substantial number of abortion seekers accessed services despite the implementation of state-level bans and restrictions.

GraFITi: Graphs for Forecasting Irregularly Sampled Time Series

Forecasting irregularly sampled time series with missing values is a crucial task for numerous real-world applications such as healthcare, astronomy, and climate sciences. State-of-the-art approaches to this problem rely on Ordinary Differential Equations (ODEs) which are known to be slow and often require additional features to handle missing values. To address this issue, we propose a novel model using Graphs for Forecasting Irregularly Sampled Time Series with missing values which we call GraFITi. GraFITi first converts the time series to a Sparsity Structure Graph which is a sparse bipartite graph, and then reformulates the forecasting problem as the edge weight prediction task in the graph. It uses the power of Graph Neural Networks to learn the graph and predict the target edge weights. GraFITi has been tested on 3 real-world and 1 synthetic irregularly sampled time series dataset with missing values and compared with various state-of-the-art models. The experimental results demonstrate that GraFITi improves the forecasting accuracy by up to 17% and reduces the run time up to 5 times compared to the state-of-the-art forecasting models.

Incomplete Contrastive Multi-View Clustering with High-Confidence Guiding

Incomplete multi-view clustering becomes an important research problem, since multi-view data with missing values are ubiquitous in real-world applications. Although great efforts have been made for incomplete multi-view clustering, there are still some challenges: 1) most existing methods didn't make full use of multi-view information to deal with missing values; 2) most methods just employ the consistent information within multi-view data but ignore the complementary information; 3) For the existing incomplete multi-view clustering methods, incomplete multi-view representation learning and clustering are treated as independent processes, which leads to performance gap. In this work, we proposed a novel Incomplete Contrastive Multi-View Clustering method with high-confidence guiding (ICMVC). Firstly, we proposed a multi-view consistency relation transfer plus graph convolutional network to tackle missing values problem. Secondly, instance-level attention fusion and high-confidence guiding are proposed to exploit the complementary information while instance-level contrastive learning for latent representation is designed to employ the consistent information. Thirdly, an end-to-end framework is proposed to integrate multi-view missing values handling, multi-view representation learning and clustering assignment for joint optimization. Experiments compared with state-of-the-art approaches demonstrated the effectiveness and superiority of our method. Our code is publicly available at https://github.com/liunian-Jay/ICMVC. The version with supplementary material can be found at http://arxiv.org/abs/2312.08697.